GNOME Bugzilla – Bug 615819
[testing] audio timestamp stream generator and detector
Last modified: 2018-11-03 13:05:39 UTC
Currently, in order to test the correctness of timestamp handling for various audio elements we only rely on the GstBuffer timestamps. The problem with this is that we rely on the fact that all elements properly handle/read/set/modify buffer timestamps properly. And that's the only thing we use to check for errors/regressions/stream-validity in unit-tests. In other words... lots of errors can be introduced which can't be detected by any other means than by human detection. In order to properly verify the correctness of audio elements, we need * an audio generating element (maybe an extra mode to audiotestsrc?) that will create a pattern *in* the stream which embeds the timestamp with a more or less good accuracy. * an audio pattern detection element that can detect those embedded timestamps in buffers and emit messages accordingly ("at this stream running time I saw this original timestamp"). That pattern should: * be more or less robust to rate/depth conversion * have a sync point that is as accurate as possible (to detect shifts by a few samples), * contain the full timestamp information within as little samples as possible, either before or after the sync point * *ideally* be resistant to various audio encoding techniques (so we can still detect the pattern after a encode/decode pass) Keywords for searching relevant documentation/material : watermarking, fingerprinting
Forgot to mention that a very simple version that doesn't support rate/depth conversion (i.e. no modifications) would be *very* helpful right now.
interface wise it could be done simillar to videosignal plugin in gst-plugins-bad. A rectangular pulse pattern would be most simple, but has bad bandwidth and is not very stable. Everything in the frequency domain is better, but more work. GSoC2011 :/
<ds> What would be a good generated waveform that would preserve timing information across an encode/decode though a typical audio encoder? <bilboed-pi> someone's been reading bugzilla :) --> r2d (~nico@vai69-5-88-183-204-163.fbx.proxad.net) has joined #theora <-- mdale has quit (Quit: Leaving.) --> githogori (~githogori@adsl-66-123-22-146.dsl.snfc21.pacbell.net) has joined #theora --> mdale (~dale@pdpc/supporter/student/mdale) has joined #theora <ds> bilboed-pi: yeah <ds> I was hoping gmaxwell would know <ds> although one might think I'd get better results in an audio coding channel <bilboed-pi> :) <gmaxwell> ds: How well does it need to preserve timing? <ds> gmaxwell: as well as possible <ds> gmaxwell: ideally, down to a few samples * bemasc notes that vinyl timecode scratching seems to work fine <gmaxwell> can you collect many samples? <ds> yes <bemasc> ds: then random noise should work fine <ds> gmaxwell: well, hundreds <ds> maybe thousands <ds> basically, a marker for a timestamp, followed by the timing information PSK encoded or similar <gmaxwell> I can give you you techniques which I know don't work through a lossy channel. :) <gmaxwell> But really a simple timing pulse should mostly survive, if you can take many of them. <ds> gmaxwell: that would be non-useless :) <ds> I assumed a frequency pulse would have accuracy problems because of the sharp transitions <bemasc> ds: why not just throw in white noise? <bilboed-pi> ds, requiring hundreds/thousands of samples to figure out the time would be fine, provided the location of that sync point can be as accurate as possible <ds> and perhaps a triangular shaped pulse would allow more accuracy <bemasc> white noise will get you sub-sample accuracy if you care <gmaxwell> http://www.kokkinizita.net/linuxaudio/ < "Jack_delay" does not work in a typical lossy compression channel. <ds> bemasc: but will the important phase information get through an audio codec? <bemasc> ds: yes. <gmaxwell> ds: usually. I'd probably bandlimit the white noise, but then you end up with problems getting exact sample accuracy. <ds> I hope nobody minds if I paste this into bugzilla <gmaxwell> No. What bugzilla thing? <ds> gstreamer bugzilla <ds> https://bugzilla.gnome.org/show_bug.cgi?id=615819 <gmaxwell> (by bandlimit I mean limit it to a range that I'm confident will get through with reasonable quality) <ds> yeah <bemasc> bandlimiting is not necessary. --> gantim_ (tag@p5B2DA506.dip.t-dialin.net) has joined #theora <gmaxwell> coding of HF through most perceptual codecs is realy noisy. <gmaxwell> By shaping the probe signal you can be subject to less of the channel SNR. <bemasc> true. It may improve performance. <gmaxwell> But yes, it's not strictly needed. <-- gantim has quit (Ping timeout: 245 seconds) --- gantim_ is now known as gantim <gmaxwell> I'd probably also try a log-sweep probe signal rather than the white noise. I suspect it might work well, but I'm a little concerned that it may trigger window switching and behave somewhat weirdly in some codecs. <gmaxwell> it's the same code to decode it in any case.
Ideally is still valid. Re-filing in -bad
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/18.