GNOME Bugzilla – Bug 777540
isomp4: Unable to create streams playable with MSE
Last modified: 2017-01-24 16:13:06 UTC
I wrote a little test case for transferring an MP4 file through websocket to the browser and then directly displaying it using Media Source Extensions cf. https://w3c.github.io/media-source/ i've applied the patch https://bugzilla.gnome.org/attachment.cgi?id=343726&action=edit but the same thing is true for using a plain mp4mux element i've generated a test videos using: gst-launch-1.0 videotestsrc num-buffers=10 ! video/x-raw,framerate=30/1,width=640,height=480 ! queue ! x264enc ! video/x-h264,profile=main,level=\(string\)3 ! mp4dashmux faststart=true streamable=true ! filesink location=video_main_3.mp4 this doesn't play correctly in the browser unless treated with the MP4Box utility using the commandline MP4Box -dash 10000 -frag 1000 -rap video_main_3.mp4 the output of the respective qtdump shows only marginal differences: 3,5c3,5 < major brand: isom < minor version: 00000000 < compatible brands: isom iso2 dash --- > major brand: iso5 > minor version: 00000001 > compatible brands: dash iso5 7,8c7,8 < size 114084 < start 1033 --- > size 114078 > start 1139 40c40 < modification_time 2017-01-20 16:20:08 (3567770408) --- > modification_time 2017-01-20 16:20:24 (3567770424) 60c60 < language 0 --- > language 21956 the different brand is probably not the only culprit.
(In reply to Andreas Frisch from comment #0) > the different brand is probably not the only culprit. Did you try it? That should be trivial to change, just replace those fourccs in the code
changing that doesn't help FF 50.1.0 still only displays a second of the stream whereas the MP4Box treated file gets played to the end. difference however is the duration field being set (not possible with live streams generated on the fly), different arrangement of fragments and the minor version 4,5c4,5 < Fragmented File: yes - duration 00:00:00.000 < 38 fragments - 0 SegmentIndexes --- > Fragmented File: yes - duration 00:01:06.666 > 68 fragments - 9 SegmentIndexes 7c7 < File Brand iso5 - version 0 --- > File Brand iso5 - version 1
We should always write the duration, that's not good otherwise. The muxer should rewrite this information when its done with every fragment. Also segment indexes might be useful to have, what exactly does it mean with that? sidx? For the minor version, that shouldn't affect anything but also trivial to check ;)
of which meaning is a duration field in a live stream? i'm not sure about the segment indexes but that looks like a possible problem. no difference with explicitely set minor version to 0x01
Every fragment has durations for itself
that's right, but we've got a single stream file, not individual segments like in hls
That doesn't matter. There will be one moof per fragment in there, and that should contain all the relevant values for the following media data of the fragment.
update: with thaytan's patch from https://bugzilla.gnome.org/show_bug.cgi?id=708221 it plays live encoded and muxed streams correctly with MSE in firefox, not yet in chromium 55
this looks strange but may be interesting: for a test file generated with $ gst-launch-1.0 videotestsrc pattern=ball num-buffers=1000 ! timeoverlay ! video/x-raw,framerate=30/1,width=1280,height=720 ! videoconvert ! queue max-size-buffers=1 ! x264enc speed-preset=superfast tune=zerolatency key-int-max=30 b-adapt=0 option-string=scenecut=0 ! video/x-h264,profile=main ! m. audiotestsrc num-buffers=1000 ! audioconvert ! faac ! queue ! m. mp4dashmux name=m streamable=true faststart=true fragment-duration=500 ! filesink location=ball-h264-main-aac-dash.mp4 remuxing throws a bunch of warnings $ MP4Box -dash 10000 -frag 1000 -rap ball-h264-main-aac-dash.mp4 [iso file] Box "...." is larger than container box [iso file] Box "avc1" size 177 invalid (read 181) [iso file] Box "...." is larger than container box [iso file] Box "mp4a" size 79 invalid (read 83) [iso file] Box "...." is larger than container box [iso file] Box "avc1" size 177 invalid (read 181) [iso file] Box "...." is larger than container box [iso file] Box "mp4a" size 79 invalid (read 83) [iso file] Box "...." is larger than container box [iso file] Box "avc1" size 177 invalid (read 181) [iso file] Box "...." is larger than container box [iso file] Box "mp4a" size 79 invalid (read 83)
Consistently 4 bytes smaller than expected, or what does that mean? Missing to include 4 (i.e. the size of the size field) in the size field for those atoms?
it's caused by this patch: b815c41b7eee708a9e6e14caadffa04839bbdecf is the first bad commit commit b815c41b7eee708a9e6e14caadffa04839bbdecf Author: Sebastian Dröge <sebastian@centricular.com> Date: Thu Sep 29 17:40:23 2016 +0300 qtmux: Write 4 bytes of zeroes at the end of the sample description extensions This is working around some broken software. https://bugzilla.gnome.org/show_bug.cgi?id=769048 :040000 040000 c9197e2189f229944005395d539bfa0099ad7fe7 fb896be57849e8516a41046e1e013d72b385bf45 M gst
Is anything working better if your revert that patch, or does it just make the output go away?
reverting that patch fixes playback in chromium
Let's see then what we can do about that. Do you have a list of other changes that were necessary for things to work for you?
just git master 494246f75cfc739cf677c10ad872c843138effb0 plus https://bugzilla.gnome.org/show_bug.cgi?id=668091#c12
commit d1ce68e840a87ecc5ee229d28db6d6a36e63290d Author: Sebastian Dröge <sebastian@centricular.com> Date: Tue Jan 24 17:59:59 2017 +0200 qtmux: Only write 4 byte zero padding to the Video Sample Description in MOV For MP4 this is not defined, and it actually breaks things for MSE in Chrome if we do this. For MOV this is required by some broken software but the official specification says it's optional: https://developer.apple.com/library/content/documentation/QuickTime/QTFF/QTFFChap3/qtff3.html https://bugzilla.gnome.org/show_bug.cgi?id=777540
No warnings in MP4Box and plays correctly in Chromium, yay \o/