GNOME Bugzilla – Bug 549940
[audio decoders] Add jitter correction for timestamps
Last modified: 2012-12-19 16:59:24 UTC
Currently the timestamping of decoded audio buffers is too simple. With some codecs, like WMA2, the incoming timestamps vary quite a lot from buffer to buffer. Ex : For two consecutive incoming buffers A and B we have : B.timestamp = A.timestamp + A.duration + jitter The proposal is to see if the new buffer is within 10ms of the expected timestamp, and if so, change the buffer timestamp.
Created attachment 117639 [details] [review] Proposed patch
Maybe I should explain a bit more why this is needed. In normal cases (playbin), the jitter correction will actually be done in the audio sinks (which allow up to an insane 500ms jitter). But in many other cases, you actually want that jitter fixed as upstream as possible. It can't be fixed in most demuxers, since they don't know the ACTUAL duration of the outgoing encoded buffers. You actually need to decode those samples to know their duration. Demuxers that handle raw audio could handle it though. So the next most upstream element where it can be fixed is... the decoder, which once it's decoded the sample, knows it's duration. This is needed for proper accurate seeking (used constantly with gnonlin and therefore jokosher/pitivi/...). Let's take the case of an asf/wma2 file. You want to seek to 1s, and let's say the demuxer is smart enough and does output the buffer that starts JUST before that position. Here are the values of the buffers that asfdemux will output: Buffer A timestamp 0:00.9000 duration 0:00.095 Buffer B timestamp 0:01.0500 duration 0:00.105 Those buffers will be received by the decoder, which will decode the first buffer, calculate the duration based on the number of samples (which is equal to 1s) see that it's not within the requested segment (starting at 0:01.00) and discard it. The next buffer commes along, is decoded, the actual duration is calculate (1s again) ... but gets clipped to 950ms and is outputted with a wrong timestamp. This is just one side effect of the jitter if its not corrected before the sinks, I'm sure some other issues will pop up too. The next buffer comes
I would like to see this in a separate element for now, ideally it goes into the decoder base class with various reconstruction and error concealment techniques but we can't really do that yet.
*** Bug 584813 has been marked as a duplicate of this bug. ***
The next timestamp here is calculated from prev_timestamp + duration? In that case we assume here that there are no rounding errors introduced by the duration or at least that they don't accumulate. IMHO this isn't the correct approach but I don't have a better idea ;)
Any news on this?
Not so sure about it anymore, it's a more generic issue than just for decoders. For example, you can't be sure whether the incoming jitter is due to network jitter or container format with low timestamp granularity. In the meantime, the easy fix is to use audiorate with a specified tolerance ,which is what encodebin does.
*** Bug 506557 has been marked as a duplicate of this bug. ***
I'm having the same issue recording from an AAC RTMP source (LBC Radio), url is rtmp://cdn-sov-2.musicradio.com:80/LiveAudio/LBC973 If I do: rtmpsrc location=%s ! decodebin2 ! qudiorate ! audioconvert ! vorbisenc ! oggmux ! filesink location=a.ogg I get jigger and crackling and all kinds of problems :( Strangely, if I change vorbisenc to lame, the problems go away. Is this a vorbisenc bug? - Any workaround?
Looks like to record from this source - rtmp://cdn-sov-2.musicradio.com:80/LiveAudio/LBC973 - I have to either do: rtmpsrc location=%s ! decodebin2 ! audioconvert ! wavenc ! decodebin ! audioconvert ! vorbisenc ! oggmux ! filesink location=a.ogg Or do: rtmpsrc location=%s ! decodebin2 ! audioconvert ! lame ! filesink location=a.mp3 Any cleaner way to do vorbis encoding? :/
Someone write a new wrapper for the audio codecs based on GstAudioDecoder/GstAudioEncoder please ;)
I believe this is handled by the audio base classes now in git master. Can this bug be closed?