GNOME Bugzilla – Bug 580005
separate caps for mp4mux?
Last modified: 2009-08-10 23:46:09 UTC
So as I have been working on Transmageddon one of the things I wanted to do was to be able to query for elements based on caps, for instance I want to be able to say give me an element that can mux 'application/mxf' and I would be able to identify an element that can mux me that. For most formats that should be fine, but I noticed that capswise we do not currently differentiate between Quicktime and MP4 container formats, but got the caps 'video/quicktime'. I suggest that mp4mux gets changed to something like video/x-mp4, and that qtdemux gets updated to also handle that as a incoming caps.
I noticed that our typefinder do not differentiate either between Quicktime and MP4. We should probably update that too, as one could have a mp4 only demuxer for instance.
Uraeus, have you see the discussion in bug #340375.
Ok, so looking at bug #340375 it seems the caps for the MPEG4 container format should maybe be application/x-iso-mp4 I guess we can fix the muxer to use this and also the demuxer, while the typefinding stuff might need to wait. Although part of me feel that mp4 container format being typefound as a quicktime file is a obvious bug and thus not something which should be kept as it is due to ABI considerations. As for the discussion in 340375 it seems to be mostly about if GStreamer typefinders should not only check for container format as they do today, but also prod inside the files, so I guess its a slightly different debate from this one.
Thiago, discussed this with Tim, Wim and Edward today. Ended up being a very wide ranging discussion and maybe something that would be natural to pick up on at the GStreamer summit this summer, but I think for now, you should just change the caps of the mp4 muxer to application/x-iso-mp4 and just leave the typefinders and demuxers alone for now.
Module: gst-plugins-bad Branch: master Commit: bb86b8ca7e9f5de4cdb48e1248a2675e4dfa49a1 URL: http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/commit/?id=bb86b8ca7e9f5de4cdb48e1248a2675e4dfa49a1 Author: Thiago Santos <thiagoss@embedded.ufcg.edu.br> Date: Fri Apr 24 18:53:36 2009 -0300 mp4mux: Changes src caps to application/x-iso-mp4 Fixes #580005 Done, can we close it?
Closing as fixed
Ok, so to make this discussion a bit more clear the current setup is: quicktime files - typefinds at video/quicktime 3gpp files - typefinds as application/x-3gp mp4 files - typefinds as video/quicktime qtdemux handles video/quicktime and application/x-3gp as its input caps qtmux outputs video/quicktime 3gpmux outputs application/x-3gp mp4mux oututs application/x-iso-mp4 Solution options for going forward - all formats share the same main name, but get separate attributes, for example: video/quicktime,type=apple video/quicktime,type=3gpp video/quicktime,type=iso Or we define 3 separate mimetypes: video/quicktime application/x-3gp application/x-iso-mp4 or application/x-mp4 The gst-typefinder, would need to keep returning video/quicktime for mp4 no matter what, and I guess the qtdemuxer would need to keep accepting application/x-3gp no matter what.
I vote for properties!
I vote for a caps attribute! Nothing else consistently makes sense and retains backwards compatibility.
If we do a caps attribute, I would call it ftyp (see http://www.ftyps.com/what.html). But then, whats the problem with adding application/x-iso-mp4 as a media type to the muxer, demuxer and typefinder?
Stefan, we can't add it to the typefinder without technically breaking ABI/API as there might be applications or plugins depending on the current behaviour. As for your ftyp suggestion I agree it seems it would make sense to use the ftyp name for the attribute and the ftyp codes for the various elements. Thiago, are you up for switching all the caps around to match this? The ftyp list is here: http://mp4ra.org/filetype.html so that the caps used for both typefinder, demuxer and muxer is: quicktime - video/quicktime,ftyp=qt$20$20 video/quicktime,ftyp=isom (or iso2?) video/quicktine,ftyp=3gg6 (I think, please check the mp4ra site) The demuxer needs to keep supporting the 'application/x-3gp' caps of course in addition to handling these 3 new subcaps.
actually for quicktime I guess we do not want two white space entries as part of the caps, so just 'qt' should be fine there
Christian, if we add a new caps field we change the caps as well. The only breakage I can see is that if app have some hardcoded mappings. E.g. plugging qtdemux only for video/quicktime and application/x-3gp. Now if we start to use application/x-iso-mp4, they would faild to plug qtdemux for them. Is that what we are worried about?
yes, that is what we are worried about (I am personally not to worried, but I can agree to the principle that it do constitute a ABI breakage). I guess the idea is that if we only add attributes the get_name function for instance will still return video/quicktime for mp4, and I assume the typefinder if extended to return video/quicktime,ftyp=isom, would still work with a demuxer offering to plug with video/quicktime without any attributes.
So I been pondering the attribute values back and forth. So my initial reaction was to use ftyp values as it would give people something they are familiar with. Then after discussing the issue with Wim and Tim in #gstreamer I came to the conclusion that it would just introduce semi-confusion like the GStreamer caps aka 'not mimetypes' does to a certain degree as we would typefind something as ftyp=qt, while its actually a ftyp=pana for example. But then after thinking some more about it, I am wondering if maybe the correct solution is to use the ftyp after all, cause in theory I guess we could want our muxer to be able to output all the 31 types of quicktime listed by mp4ra. And we might at some point want separate typefinders/typefinding for all those 31 types. And at that point using our own values might start to be even more confusing/non-optimal than adopting the ftyp here. Wim mentioned that there might be Quicktime files without a ftyp, but I guess we could sort that by adding a 'none' subcap or something?
Ok, had one more discussion about this in #gstreamer. wtay, MikeS, __tim, mnauw, thiagoss and myself taking part. Seems the use of ftyp values are not to popular due to being ugly and maybe to specific for what we are ever going to do. Some of them might be better done as element settings, like the 3gpp subtypes for instance if we ever support them. So we are back to the orginal suggestion for subcaps naming, with using the word variant instead of type, cause Mike and Tim likes it more, and also cause we have precedence for that word in existing elements (h263). video/quicktime,variant=apple video/quicktime,variant=3gpp video/quicktime,variant=iso
Just for the record, my issue here is that e.g. m4a files are not video/quicktime, especially not "video". So please tag the changes with big fat "FIXME-0.11".
Module: gst-plugins-bad Branch: master Commit: 76aa98549a841d8939859c5a869e66805cdbaa5e URL: http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/commit/?id=76aa98549a841d8939859c5a869e66805cdbaa5e Author: Thiago Santos <thiagoss@embedded.ufcg.edu.br> Date: Thu Apr 30 14:43:36 2009 -0300 Pushed the changes to qtmux/mp4mux/gppmux, should typefinders be updated as well?
The demuxer needs to support these too, so you can do mux ! demux. As for the typefinders I don't know exactly how things work in terms of keeping things ABI compatible. The 3gpp typefinder can not change as it is not using video/quickime, that is for sure. I guess the mp4 and quicktime typefinder can add these new subcaps though? Can someone else please confirm or reject that?
The demuxers will be fine, they don't specify the variant so they take everything. Therefore the typefinders also don't need to be changed.