After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 742643 - isomp4: Add support for Opus
isomp4: Add support for Opus
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
git master
Other Linux
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Reported: 2015-01-09 13:17 UTC by Sebastian Dröge (slomo)
Modified: 2018-11-03 14:56 UTC
See Also:
GNOME target: ---
GNOME version: ---

work in progress patch (2.87 KB, patch)
2015-11-16 15:13 UTC, Luis de Bethencourt
none Details | Review
isomp4: add support for Opus in mp4mux (6.11 KB, patch)
2015-11-17 15:22 UTC, Luis de Bethencourt
none Details | Review
now we read the pre_skip and gain from the OpusHeader (6.87 KB, patch)
2015-11-17 17:05 UTC, Luis de Bethencourt
none Details | Review
[WIP] isomp4: add support for Opus in mp4mux (7.08 KB, patch)
2015-11-18 16:58 UTC, Luis de Bethencourt
committed Details | Review

Description Sebastian Dröge (slomo) 2015-01-09 13:17:13 UTC
It's official now:
Comment 1 Sebastian Dröge (slomo) 2015-11-03 19:08:16 UTC
With the new helper functions from pbutils, this should be rather simple now
Comment 2 Luis de Bethencourt 2015-11-04 15:55:21 UTC
This looks very interesting.
Comment 3 Reynaldo H. Verdejo Pinochet 2015-11-14 23:44:55 UTC
Luis: are you still working on this?
Comment 4 Luis de Bethencourt 2015-11-15 01:24:11 UTC
Yes I am :)
Comment 5 Luis de Bethencourt 2015-11-16 15:13:28 UTC
Created attachment 315683 [details] [review]
work in progress patch

Just a status update since Sebastian and Reynaldo are interested in this bug.
Comment 6 Luis de Bethencourt 2015-11-16 15:31:14 UTC
For clarity, the above patch is unfinished and just sharing the current progress state.

Still pending is populating the OpusSampleEntry box.
Comment 7 Luis de Bethencourt 2015-11-17 15:22:37 UTC
Created attachment 315759 [details] [review]
isomp4: add support for Opus in mp4mux

To test this patch:
gst-launch-1.0 videotestsrc num-buffers=500 ! video/x-raw,width=320,height=240 ! x264enc ! mp4mux name=mux ! filesink location=test.mp4 audiotestsrc wave=5 num-buffers=1000 ! audio/x-raw,channels=2,rate=48000 ! audioconvert ! opusenc ! mux.

And then play the generated test.mp4 file with git vlc.

I will add playback support for Opus in qtdemux in the next few days.
Comment 8 Sebastian Dröge (slomo) 2015-11-17 15:27:19 UTC
Review of attachment 315759 [details] [review]:

::: gst/isomp4/gstqtmux.c
@@ +3685,3 @@
+    ext_atom = build_opus_extension (rate, channels, mapping_family,
+        stream_count, coupled_count, channel_mapping, 0, 0);

These two zeroes here are not good :) The first zero would ideally be calculated from the GstAudioClippingMeta of the first buffers, the second we might want to add to the caps. Alternatively you can require a OpusHead streamheader in the caps (like oggmux/matroskamux) and read the two values from there. Transmuxing from e.g. RTP would require using opusparse then.

::: gst/isomp4/gstqtmuxmap.c
@@ +136,3 @@
+  "audio/x-opus, " \
+  "channels = (int) [1, 8], " \
+  "channel-mapping-family = (int) {0, 1};"

In theory we should also support other families and more channels
Comment 9 Luis de Bethencourt 2015-11-17 17:05:56 UTC
Created attachment 315775 [details] [review]
now we read the pre_skip and gain from the OpusHeader

Thanks for the review Sebastian!

Fixed the two issues you mentioned.
Comment 10 Sebastian Dröge (slomo) 2015-11-18 07:36:30 UTC
Review of attachment 315775 [details] [review]:

Main question remaining now is if anything special with the timestamps is needed like with Matroska, because of the pre-skip and padding for the last frame. Our timestamps will start at 0 and duration of the first frame will be shorter, and then the last frame will also have a shorter duration. For Matroska for example, the container wants timestamps/durations that include the pre-skip and padding.

::: gst/isomp4/gstqtmux.c
@@ +3686,3 @@
+        !GST_VALUE_HOLDS_ARRAY (streamheader) ||
+        gst_value_array_get_size (streamheader) == 0) {
+      GST_ERROR ("no streamheader field in caps %" GST_PTR_FORMAT, caps);


@@ +3687,3 @@
+        gst_value_array_get_size (streamheader) == 0) {
+      GST_ERROR ("no streamheader field in caps %" GST_PTR_FORMAT, caps);
+      goto refuse_caps;

We could also just make this a warning and assume that the two values are 0 and just use the caps instead
Comment 11 Luis de Bethencourt 2015-11-18 16:58:10 UTC
Created attachment 315847 [details] [review]
[WIP] isomp4: add support for Opus in mp4mux

Latest version. Applied the fix to parsing the caps as a fallback when there isn't a streamheader.

When generating files with audiotestsrc this works. When creating them by transcoding a file, vlc complains that the Opus stream is corrupted. Surely to do with what Sebastian points out about timestamps.

Will continue debugging this tomorrow.
Comment 12 Luis de Bethencourt 2015-11-18 16:58:38 UTC
Review of attachment 315847 [details] [review]:

Needs work.
Comment 13 Luis de Bethencourt 2015-11-19 17:13:29 UTC
Review of attachment 315847 [details] [review]:

commit 5ed8cba024c001e8d40f4189f0e5168893e772aa
Author: Luis de Bethencourt <>
Date:   Mon Nov 16 13:26:50 2015 +0000

    isomp4: add support for Opus in mp4mpux

    Add support for muxing MP4 files containing Opus. Based on the spec
    detailed here:

VLC was tricking me by only accepting mono or stereo streams. Anything else will be considered a corrupt stream. Eventhough it isn't.
Comment 14 Luis de Bethencourt 2015-11-19 17:14:22 UTC
Now I am going to look into supporting demuxing of Opus in qtdemux.

After that I will revisit the possibility of having to shift timestamps based on the pre_skip.
Comment 15 Luis de Bethencourt 2015-11-26 21:56:41 UTC
commit a400d504ca25ba3812a563a4cac94ef2bf910b2a
Author: Luis de Bethencourt <>
Date:   Thu Nov 26 21:46:11 2015 +0000

    qtdemux: add support for Opus

    Add support for demuxing Opus encapsulated in MP4 files, based on the
    following spec:
Comment 16 Luis de Bethencourt 2015-11-26 21:57:27 UTC
Now we can mux and demux.

Demuxer work has been tested in mono, stereo and 5.1 files.

Remaining work is to revisit the possibility of having to shift timestamps based on the pre_skip.
Comment 17 Sebastian Dröge (slomo) 2015-11-27 02:58:47 UTC
And actual *handling* of pre-skip and the end padding in the demuxer and muxer, that is putting the GstAudioClippingMeta on buffers or reading it from buffers.

If you need to shift timestamps or not is a separate issue to this.
Comment 18 Luis de Bethencourt 2015-11-27 11:58:51 UTC
Roger that.

Thanks for the help Sebastian :)

Will be on this very soon.
Comment 19 Sebastian Dröge (slomo) 2015-12-09 10:01:50 UTC
What's the progress here?
Comment 20 Luis de Bethencourt 2015-12-09 10:15:46 UTC
Hi Sebastian.

Sorry, urgent things got in the way of important ones. Will have time to put the GstAudioClippingMeta in the next few days for sure.
Comment 21 Luis de Bethencourt 2015-12-10 18:18:58 UTC
The spec [0] defines that:
"Due to the priming samples (or the padding at the beginning) derived from the pre-roll for the startup and the padded samples at the end, we need trim from media to get the actual duration. An edit in the Edit List Box can achieve this demand, and the Edit Box and the Edit List Box shall be present."

In other words, the pre_skip should be the media_start value of the elst_entry.

Which is set in gst_qt_mux_update_edit_lists () [0], which if qtpad->dts_adjustment is > 0, passes (qtpad->first_ts - qtpad->first_dts) as media_start.

Now, the question is how do I make the pre_skip, gathered in gst_qt_mux_audio_sink_set_caps(), available in gst_qt_mux_add_buffer() to be able to set dts_adjustment and first_dts in?

Comment 22 Sebastian Dröge (slomo) 2018-05-04 12:56:08 UTC
Let's reassign to the default assignee then, there does not seem to be any progress here :)
Comment 23 GStreamer system administrator 2018-11-03 14:56:54 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: