After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 791099 - opusparse: subtract clipped audio to buffer duration
opusparse: subtract clipped audio to buffer duration
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other All
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2017-12-01 20:28 UTC by Alicia Boya García
Modified: 2018-11-03 14:15 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
opusparse: subtract clipped audio to buffer duration (1.17 KB, patch)
2018-02-16 23:17 UTC, Alicia Boya García
none Details | Review
opusparse: subtract clipped audio to buffer duration (1.44 KB, patch)
2018-04-12 21:41 UTC, Alicia Boya García
none Details | Review
opusparse: subtract clipped audio to buffer duration (2.41 KB, patch)
2018-04-16 16:19 UTC, Alicia Boya García
none Details | Review

Description Alicia Boya García 2017-12-01 20:28:04 UTC
opusparse is not subtracting codec delay from the buffer duration.

Generate test vector:
$ gst-launch-1.0 audiotestsrc num-buffers=1 samplesperbuffer=24000 ! audio/x-raw, rate=48000 ! opusenc ! identity error-after=2 ! webmmux ! filesink location=/tmp/audio-clipping-test-vector.webm

Test:
$ gst-launch-1.0 -v pushfilesrc location=/tmp/audio-clipping-test-vector.webm \
  ! matroskademux ! identity name="after_matroskademux" silent=false \
  ! opusparse     ! identity name="after_opusparse" silent=false \
  ! opusdec       ! identity name="after_opusdec" silent=false \
  ! fakesink 2>&1 |grep chain

/GstPipeline:pipeline0/GstIdentity:after_matroskademux: last-message = chain   ******* (after_matroskademux:sink) (160 bytes, dts: none, pts: 0:00:00.000000000, duration: 0:00:00.013500000, offset: -1, offset_end: -1, flags: 00004040 discont tag-memory , meta: GstAudioClippingMeta) 0x7f13c800a060
/GstPipeline:pipeline0/GstIdentity:after_opusparse: last-message = chain   ******* (after_opusparse:sink) (160 bytes, dts: 0:00:00.000000000, pts: 0:00:00.000000000, duration: 0:00:00.020000000, offset: 20000000, offset_end:  960, flags: 00004040 discont tag-memory , meta: GstAudioClippingMeta) 0x7f13c800a060
/GstPipeline:pipeline0/GstIdentity:after_opusdec: last-message = chain   ******* (after_opusdec:sink) (1296 bytes, dts: none, pts: 0:00:00.000000000, duration: 0:00:00.013500000, offset: -1, offset_end: -1, flags: 00000040 discont , meta: none) 0x7f13c800ab00

Note the inconsistency in durations and GstAudioClippingMeta:

Expected*:

matroskademux ------------> opusparse -----------> opusdec ------------>
                 13.5ms                  13.5ms                13.5ms
                clip meta               clip meta           no clip meta

Actual:

matroskademux ------------> opusparse -----------> opusdec ------------>
                 13.5ms                  20.0ms                13.5ms
                clip meta               clip meta           no clip meta

* I'm supposing that GstBuffer durations should already have GstAudioClippingMeta subtracted. I could not find any hints in the documentation one way or the other, so I'm picking sides with matroskademux because is gst-plugins-good. In any case, the current behavior is definitively inconsistent because matroskademux is subtracting them while opusparse is not.
Comment 1 Alicia Boya García 2018-02-16 23:17:04 UTC
An additional reason for subtracting clipped audio: It makes sense that the user-visible duration of the GstBuffer remains the same whether the audio is compressed or raw.

Also, some applications (like WebKit MediaSource implementation) need to know whether two audio frames overlap before decoding or playing them, so consistency is also a plus there.
Comment 2 Alicia Boya García 2018-02-16 23:17:30 UTC
Created attachment 368449 [details] [review]
opusparse: subtract clipped audio to buffer duration
Comment 3 Alicia Boya García 2018-04-12 21:41:08 UTC
Created attachment 370875 [details] [review]
opusparse: subtract clipped audio to buffer duration

This is consistent with other elements such as matroskademux and also
ensures that duration does not change after decoding the audio.

Correct duration is important for applications that need to account for
frame overlaps, such as in Media Source Extensions enabled browsers.
Comment 4 Sebastian Dröge (slomo) 2018-04-16 08:27:55 UTC
Comment on attachment 370875 [details] [review]
opusparse: subtract clipped audio to buffer duration

opusparse should probably additionally take the pre-skip out of the streamheader in the caps, if given. So that inputs without meta are still handled correctly if a streamheader is provided.

Apart from that this looks correct to me
Comment 5 Alicia Boya García 2018-04-16 16:19:41 UTC
Created attachment 370996 [details] [review]
opusparse: subtract clipped audio to buffer duration

This makes opusparse take into account audio priming for the first audio
frame and end clipping for the last frame.

Audio priming is determined by `parse->pre_skip`, whose value is read
from either the header frame (usually provided in the caps) or -- in its
absence, from GstAudioClippingMeta.start.

End clipping is read from GstAudioClippingMeta.end.
Comment 6 Alicia Boya García 2018-05-17 16:59:11 UTC
Could you please give a look at this? The new patch handles pre-skip in both streamheader and GstAudioClippingMeta.
Comment 7 GStreamer system administrator 2018-11-03 14:15:57 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/635.