After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 549940 - [audio decoders] Add jitter correction for timestamps
[audio decoders] Add jitter correction for timestamps
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-libav
git master
Other Linux
: Normal normal
: NONE
Assigned To: GStreamer Maintainers
GStreamer Maintainers
: 506557 584813 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2008-08-30 14:17 UTC by Edward Hervey
Modified: 2012-12-19 16:59 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Proposed patch (1.69 KB, patch)
2008-08-30 14:18 UTC, Edward Hervey
none Details | Review

Description Edward Hervey 2008-08-30 14:17:30 UTC
Currently the timestamping of decoded audio buffers is too simple.

With some codecs, like WMA2, the incoming timestamps vary quite a lot from buffer to buffer. Ex : For two consecutive incoming buffers A and B we have :

B.timestamp = A.timestamp + A.duration + jitter

The proposal is to see if the new buffer is within 10ms of the expected timestamp, and if so, change the buffer timestamp.
Comment 1 Edward Hervey 2008-08-30 14:18:02 UTC
Created attachment 117639 [details] [review]
Proposed patch
Comment 2 Edward Hervey 2008-08-30 14:35:39 UTC
Maybe I should explain a bit more why this is needed.

In normal cases (playbin), the jitter correction will actually be done in the audio sinks (which allow up to an insane 500ms jitter).

 But in many other cases, you actually want that jitter fixed as upstream as possible.
 It can't be fixed in most demuxers, since they don't know the ACTUAL duration of the outgoing encoded buffers. You actually need to decode those samples to know their duration. Demuxers that handle raw audio could handle it though.

 So the next most upstream element where it can be fixed is... the decoder, which once it's decoded the sample, knows it's duration.

 This is needed for proper accurate seeking (used constantly with gnonlin and therefore jokosher/pitivi/...).

 Let's take the case of an asf/wma2 file. You want to seek to 1s, and let's say the demuxer is smart enough and does output the buffer that starts JUST before that position.
 Here are the values of the buffers that asfdemux will output:
 Buffer A timestamp 0:00.9000 duration 0:00.095
 Buffer B timestamp 0:01.0500 duration 0:00.105

 Those buffers will be received by the decoder, which will decode the first buffer, calculate the duration based on the number of samples (which is equal to 1s) see that it's not within the requested segment (starting at 0:01.00) and discard it.
 The next buffer commes along, is decoded, the actual duration is calculate (1s again) ... but gets clipped to 950ms and is outputted with a wrong timestamp.

 This is just one side effect of the jitter if its not corrected before the sinks, I'm sure some other issues will pop up too.
 The next buffer comes
Comment 3 Wim Taymans 2008-09-05 14:39:28 UTC
I would like to see this in a separate element for now, ideally it goes into the decoder base class with various reconstruction and error concealment techniques but we can't really do that yet.

Comment 4 Wim Taymans 2009-06-04 14:34:24 UTC
*** Bug 584813 has been marked as a duplicate of this bug. ***
Comment 5 Sebastian Dröge (slomo) 2009-07-30 11:28:18 UTC
The next timestamp here is calculated from prev_timestamp + duration? In that case we assume here that there are no rounding errors introduced by the duration or at least that they don't accumulate. IMHO this isn't the correct approach but I don't have a better idea ;)
Comment 6 Sebastian Dröge (slomo) 2011-05-09 09:05:19 UTC
Any news on this?
Comment 7 Edward Hervey 2011-05-09 10:48:26 UTC
Not so sure about it anymore, it's a more generic issue than just for decoders.

For example, you can't be sure whether the incoming jitter is due to network jitter or container format with low timestamp granularity.

In the meantime, the easy fix is to use audiorate with a specified tolerance ,which is what encodebin does.
Comment 8 Sebastian Dröge (slomo) 2011-05-19 06:14:51 UTC
*** Bug 506557 has been marked as a duplicate of this bug. ***
Comment 9 Roman Gaufman 2011-07-02 15:20:13 UTC
I'm having the same issue recording from an AAC RTMP source (LBC Radio), url is rtmp://cdn-sov-2.musicradio.com:80/LiveAudio/LBC973

If I do:

rtmpsrc location=%s ! decodebin2 ! qudiorate ! audioconvert ! vorbisenc ! oggmux ! filesink location=a.ogg

I get jigger and crackling and all kinds of problems :(

Strangely, if I change vorbisenc to lame, the problems go away. Is this a vorbisenc bug? - Any workaround?
Comment 10 Roman Gaufman 2011-07-02 15:44:45 UTC
Looks like to record from this source - rtmp://cdn-sov-2.musicradio.com:80/LiveAudio/LBC973
 -  I have to either do:

rtmpsrc location=%s ! decodebin2 ! audioconvert ! wavenc ! decodebin ! audioconvert ! vorbisenc ! oggmux ! filesink location=a.ogg

Or do:

rtmpsrc location=%s ! decodebin2 ! audioconvert ! lame ! filesink location=a.mp3

Any cleaner way to do vorbis encoding? :/
Comment 11 Tim-Philipp Müller 2012-02-24 17:39:57 UTC
Someone write a new wrapper for the audio codecs based on GstAudioDecoder/GstAudioEncoder please ;)
Comment 12 Tim-Philipp Müller 2012-12-13 23:48:35 UTC
I believe this is handled by the audio base classes now in git master. Can this bug be closed?