GNOME Bugzilla – Bug 791099
opusparse: subtract clipped audio to buffer duration
Last modified: 2018-11-03 14:15:57 UTC
opusparse is not subtracting codec delay from the buffer duration. Generate test vector: $ gst-launch-1.0 audiotestsrc num-buffers=1 samplesperbuffer=24000 ! audio/x-raw, rate=48000 ! opusenc ! identity error-after=2 ! webmmux ! filesink location=/tmp/audio-clipping-test-vector.webm Test: $ gst-launch-1.0 -v pushfilesrc location=/tmp/audio-clipping-test-vector.webm \ ! matroskademux ! identity name="after_matroskademux" silent=false \ ! opusparse ! identity name="after_opusparse" silent=false \ ! opusdec ! identity name="after_opusdec" silent=false \ ! fakesink 2>&1 |grep chain /GstPipeline:pipeline0/GstIdentity:after_matroskademux: last-message = chain ******* (after_matroskademux:sink) (160 bytes, dts: none, pts: 0:00:00.000000000, duration: 0:00:00.013500000, offset: -1, offset_end: -1, flags: 00004040 discont tag-memory , meta: GstAudioClippingMeta) 0x7f13c800a060 /GstPipeline:pipeline0/GstIdentity:after_opusparse: last-message = chain ******* (after_opusparse:sink) (160 bytes, dts: 0:00:00.000000000, pts: 0:00:00.000000000, duration: 0:00:00.020000000, offset: 20000000, offset_end: 960, flags: 00004040 discont tag-memory , meta: GstAudioClippingMeta) 0x7f13c800a060 /GstPipeline:pipeline0/GstIdentity:after_opusdec: last-message = chain ******* (after_opusdec:sink) (1296 bytes, dts: none, pts: 0:00:00.000000000, duration: 0:00:00.013500000, offset: -1, offset_end: -1, flags: 00000040 discont , meta: none) 0x7f13c800ab00 Note the inconsistency in durations and GstAudioClippingMeta: Expected*: matroskademux ------------> opusparse -----------> opusdec ------------> 13.5ms 13.5ms 13.5ms clip meta clip meta no clip meta Actual: matroskademux ------------> opusparse -----------> opusdec ------------> 13.5ms 20.0ms 13.5ms clip meta clip meta no clip meta * I'm supposing that GstBuffer durations should already have GstAudioClippingMeta subtracted. I could not find any hints in the documentation one way or the other, so I'm picking sides with matroskademux because is gst-plugins-good. In any case, the current behavior is definitively inconsistent because matroskademux is subtracting them while opusparse is not.
An additional reason for subtracting clipped audio: It makes sense that the user-visible duration of the GstBuffer remains the same whether the audio is compressed or raw. Also, some applications (like WebKit MediaSource implementation) need to know whether two audio frames overlap before decoding or playing them, so consistency is also a plus there.
Created attachment 368449 [details] [review] opusparse: subtract clipped audio to buffer duration
Created attachment 370875 [details] [review] opusparse: subtract clipped audio to buffer duration This is consistent with other elements such as matroskademux and also ensures that duration does not change after decoding the audio. Correct duration is important for applications that need to account for frame overlaps, such as in Media Source Extensions enabled browsers.
Comment on attachment 370875 [details] [review] opusparse: subtract clipped audio to buffer duration opusparse should probably additionally take the pre-skip out of the streamheader in the caps, if given. So that inputs without meta are still handled correctly if a streamheader is provided. Apart from that this looks correct to me
Created attachment 370996 [details] [review] opusparse: subtract clipped audio to buffer duration This makes opusparse take into account audio priming for the first audio frame and end clipping for the last frame. Audio priming is determined by `parse->pre_skip`, whose value is read from either the header frame (usually provided in the caps) or -- in its absence, from GstAudioClippingMeta.start. End clipping is read from GstAudioClippingMeta.end.
Could you please give a look at this? The new patch handles pre-skip in both streamheader and GstAudioClippingMeta.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/635.