GNOME Bugzilla – Bug 727305
matroskademux: complete support for A_OPUS (SeekPreRoll, CodecDelay and DiscardPadding)
Last modified: 2016-02-03 10:32:12 UTC
Tools support muxing Opus audio in Matroska (MKV) files since last year [1]. I have created a test file and was able to play it on my Nexus 7 tablet using MX, however the same file does not work on Totem 3.12: --- test.mkv requires an additional plugin to decode this file The following plugin is required: audio/x-unknown decoder Do you want to search for this now? --- The mkvmerge tool shows this: $ mkvmerge -i test.mkv File 'mask-opus.mkv': container: Matroska Track ID 0: video (MPEG-4p10/AVC/h.264) Track ID 1: audio (Opus) [1] http://lists.matroska.org/pipermail/matroska-users/2013-September/006883.html
Do you have an example file?
Created attachment 273263 [details] Test file Here's a small test file I created based on some video from archive.org, it plays correctly using mplayer.
I confirm. No decoder available for type 'audio/x-unknown, codec-id=(string)A_OPUS It seems a appropriate mapping of A_OPUS is required in matroska demux.
commit b158a1c068460f79a6753c9b80eda981c8172030 Author: Tim-Philipp Müller <tim@centricular.com> Date: Sun Mar 30 00:31:11 2014 +0000 matroska-demux: add mapping for Opus audio codec https://bugzilla.gnome.org/show_bug.cgi?id=727305
This is not enough yet: https://wiki.xiph.org/MatroskaOpus See the SeekPreRoll, CodecDelay and DiscardPadding EBML elements.
Thanks for taking a look everyone. So according to that wiki page, the spec for Opus ind MKV is not even finalized? However, multiple tools officially support this... Maybe the wiki is outdated?
The wiki is unfortunately outdated, I think the mkvtoolnix code is the current "spec".
Also: commit b7413279d9ff2956b04360f40a806557f3c18b9c Author: Vincent Penquerc'h <vincent.penquerch@collabora.co.uk> Date: Wed Dec 17 17:36:18 2014 +0000 matroska: mux/demux the OpusHead header This is meant to be so (https://wiki.xiph.org/MatroskaOpus - while it is marked as a draft, this part was confirmed to be correct on IRC), and allows one to determine whether a demuxed stream is multistream or not, and thus set the multistream caps field accordingly. In turn, this means downstream does not have to guess. https://bugzilla.gnome.org/show_bug.cgi?id=740744 But those other fields are still missing I suppose?
(In reply to comment #8) > But those other fields are still missing I suppose? Yes, and we actually need to use them. I think they would change segment or timestamp information.
Created attachment 314712 [details] [review] matroskamux: Write CodecDelay, DiscardPadding and SeekPreroll for Opus
Created attachment 314714 [details] [review] matroskamux: Write CodecDelay, DiscardPadding and SeekPreroll for Opus
Created attachment 314717 [details] [review] matroskamux: Write CodecDelay, DiscardPadding and SeekPreroll for Opus
Created attachment 314722 [details] [review] matroskamux: Write CodecDelay, DiscardPadding and SeekPreroll for Opus And also adjust timestamps and durations according to the codec delay, both should include it for whatever reason.
Created attachment 314723 [details] [review] matroskademux: Parse and handle CodecDelay, SeekPreroll and DiscardPadding
Merged
It looks like I found a case where it still does not work (using GIT master of today). The attached sample file contains VP8 video and OPUS audio: ffprobe 2016-02-03_-_09-54-34.mkv ffprobe version git-2016-01-22-74105fc Copyright (c) 2007-2016 the FFmpeg developers [...] Input #0, matroska,webm, from '/storage/2016-02-03_-_09-54-34_-_Medialab-6Plus.mkv': Metadata: encoder : GStreamer matroskamux version 1.7.1.1 creation_time : 2016-02-03 08:54:35 Duration: N/A, start: 0.024000, bitrate: N/A Stream #0:0(eng): Video: vp8, yuv420p, 1280x720, SAR 1:1 DAR 16:9, 1k tbr, 1k tbn, 1k tbc (default) Metadata: title : Video Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default) Metadata: title : Audio The source of the clip is a stream generated by Google's libwebrtc for Ios received over RTP, depayloaded and muxed using matroskamux. The file plays just fine with ffplay, but playbin uri=file:///storage/2016-02-03_-_09-54-34.mkv shows the following output: Setting pipeline to PAUSED ... /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0: ring-buffer-max-size = 0 /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0: buffer-size = -1 /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0: buffer-duration = -1 /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0: use-buffering = false /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0: download = false /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0: uri = file:///storage/2016-02-03_-_09-54-34.mkv /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0: connection-speed = 0 /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0: source = "\(GstFileSrc\)\ source" Pipeline is PREROLLING ... /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = "NULL" /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = "NULL" Missing element: audio/x-unknown decoder /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5 /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0 /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152 /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstPad:sink_0: caps = "video/x-vp8\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstPad:src_0: caps = "video/x-vp8\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstVP8Dec:vp8dec0.GstPad:sink: caps = "video/x-vp8\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstVP8Dec:vp8dec0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5 WARNING: from element /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0: No decoder available for type 'audio/x-unknown, codec-id=(string)A_OPUS'. /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0 Additional debug info: gsturidecodebin.c(939): unknown_type_cb (): /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0 /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152 /GstPlayBin:playbin0/GstInputSelector:inputselector0.GstSelectorPad:sink_0: always-ok = false /GstPlayBin:playbin0/GstInputSelector:inputselector0.GstSelectorPad:sink_0: active = true /GstPlayBin:playbin0/GstInputSelector:inputselector0: active-pad = "\(GstSelectorPad\)\ sink_0" /GstPlayBin:playbin0/GstInputSelector:inputselector0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink.GstGhostPad:video_sink.GstProxyPad:proxypad5: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstStreamSynchronizer:streamsynchronizer0.GstPad:src_0: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vdbin.GstGhostPad:sink.GstProxyPad:proxypad9: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vdbin/GstVideoConvert:vdconv.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vdbin/GstDeinterlace:deinterlace.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vdbin.GstGhostPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin.GstGhostPad:sink.GstProxyPad:proxypad8: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstQueue:vqueue.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin.GstGhostPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin.GstGhostPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstPlaySinkVideoConvert:vconv.GstGhostPad:sink.GstProxyPad:proxypad6: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstPlaySinkVideoConvert:vconv.GstGhostPad:sink.GstProxyPad:proxypad6: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstPlaySinkVideoConvert:vconv/GstVideoConvert:conv.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstPlaySinkVideoConvert:vconv/GstVideoScale:scale.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstPlaySinkVideoConvert:vconv.GstGhostPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstXvImageSink:xvimagesink0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstPlaySinkVideoConvert:vconv.GstGhostPad:src.GstProxyPad:proxypad7: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstPlaySinkVideoConvert:vconv/GstVideoScale:scale.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstPlaySinkVideoConvert:vconv/GstVideoConvert:conv.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstPlaySinkVideoConvert:vconv.GstGhostPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)mpeg2\,\ colorimetry\=\(string\)bt709\,\ framerate\=\(fraction\)0/1" /GstPlayBin:playbin0/GstInputSelector:inputselector0.GstSelectorPad:sink_0: tags = "taglist\,\ video-codec\=\(string\)\"VP8\\\ video\"\;" /GstPlayBin:playbin0/GstInputSelector:inputselector0.GstSelectorPad:sink_0: tags = "taglist\,\ video-codec\=\(string\)\"VP8\\\ video\"\,\ container-format\=\(string\)Matroska\;" /GstPlayBin:playbin0/GstInputSelector:inputselector0.GstSelectorPad:sink_0: tags = "taglist\,\ video-codec\=\(string\)\"On2\\\ VP8\"\,\ container-format\=\(string\)Matroska\;" Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstSystemClock Notice the: WARNING: from element /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0: No decoder available for type 'audio/x-unknown, codec-id=(string)A_OPUS'. Is this an issue in matroskamux or matroskademux? I suspect the latter but maybe something goes haywire while muxing.
Created attachment 320321 [details] Test file which still fails to demux.
Arjen, would probably be best if you just filed a new bug for that, or cloned this one.
Done, see: https://bugzilla.gnome.org/show_bug.cgi?id=761489