GNOME Bugzilla – Bug 784969
matroskamux: Should use SimpleBlocks for Opus and probably other audio codecs
Last modified: 2017-08-11 08:06:26 UTC
> gst-launch-1.0 audiotestsrc num-buffers=1000 ! opusenc ! matroskamux ! filesink location=test.mka This plays fine in GStreamer and ffplay, but VLC only plays silence. > gst-launch-1.0 audiotestsrc num-buffers=1000 ! vorbisenc ! matroskamux ! filesink location=test.mka This plays fine everywhere. > ffmpeg -i test.mka -acodec opus test2.mka This plays fine everywhere too. We must be doing something weird here that confuses VLC.
WRN00C: Unknown element in TrackEntry [56][BB] at 421 (size 4 total 7) WRN00C: Unknown element in TrackEntry [56][AA] at 428 (size 3 total 6) WRN201: Unknown 'BlockVirtual' for read profile 'matroska v2' in BlockGroup at 211505 vlc does do a short 'blip' at the very end ;)
(In reply to Tim-Philipp Müller from comment #1) > WRN00C: Unknown element in TrackEntry [56][BB] at 421 (size 4 total 7) > WRN00C: Unknown element in TrackEntry [56][AA] at 428 (size 3 total 6) SeekPreroll and CodecDelay also written by ffmpeg (with the same values) and required for Opus. > WRN201: Unknown 'BlockVirtual' for read profile 'matroska v2' in BlockGroup > at 211505 Where do you see that? We don't write BlockVirtual But we write real BlockGroups with Blocks, while ffmpeg writes SimpleBlocks (except for the last). We should do that too as it causes overhead to not use SimpleBlocks.
The problem here is that VLC apparently only likes Opus in SimpleBlocks. But we can't write SimpleBlocks here as we don't know the duration of all buffers. We could assume that duration of the first to be the default duration, and in 99% of the cases that would work, but there's no guarantee. When making matroskamux output SimpleBlocks, VLC plays this just fine. This seems like a bug in VLC (nothing requires SimpleBlocks to be used for Opus in the spec), and an inefficiency on our side
See https://trac.videolan.org/vlc/ticket/18545 for the VLC bug report.
Created attachment 355802 [details] [review] matroskamux: For audio tracks, take the default duration from the first buffer ... if we don't have any better idea from the caps. This allows writing SimpleBlocks for a majority of audio streams where the duration of frames is usually fixed. And as a side effect, allows VLC to play streams with Opus as it only works with SimpleBlocks currently: https://trac.videolan.org/vlc/ticket/18545
This makes about 6% difference in file size for a simple "audiotestsrc ! opusenc ! matroskamux" pipeline.
While it's probably a bit optimistic to assume that upstream puts correct duration on buffers, it shouldn't cause problems if it is incorrect. We would just put (wrong) durations on each Block again then. A future patch could extract codec specific durations from buffers directly by parsing, but I'd postpone that for later when it's actually needed.
If upstream puts incorrect durations on there, that's an upstream bug then. As long as it works if upstream doesn't put durations on the buffers.
It will work as good as before the patch then (not using SimpleBlocks but full BlockGroups/Blocks)
Attachment 355802 [details] pushed as 317d338 - matroskamux: For audio tracks, take the default duration from the first buffer
Not sure if this should also go into 1.12. It reduces muxing overhead quite a bit and fixes playback in VLC (due to a VLC bug), but OTOH could also have unforeseen side effects.
I wonder if it should at least check the assumptions it makes, and fallback to full blocks if the duration is not the same every time?