GNOME Bugzilla – Bug 750449
Dynamic Resolution Change does not work with qtdemux + h264parse
Last modified: 2015-06-13 20:10:07 UTC
I am working on an h264 decoder. I was trying to playback a stream in which resolution changes after 2 secs. The stream is located at http://kakao.ro/public/VLCDynamicResolutionSamples/sample.mp4 However, the set_format function of the decoder is not called with new height and new width. I examined the source code of h264parse. H264parse parses the new height and width properly from the sps. However, in the function gst_h264_parse_update_src_caps, if height and width are set on the sink pad of h264parse, it copies this height and width and calls the gst_pad_set_caps and not the parsed height and width. I then looked at qtdemux source code. As far as I understood, qtdemux parses the height and width at the beginning of the stream and sets this on sink pad of h264parse. After that, it does not look for resolution changes and hence does not set new caps on the sink pad of h264parse. I tried this stream with a different container (MPEG ts). In this case, the demuxer does not set the height and width on h264parse sinkpad and hence h264parse will always set new height and width on its source pad.
Does ISOMP4 spec even allow that ? (this is a real question, I always thought this wasn't supported by the format)
They have to be separate tracks, and switched at the decoder level, which we probably don't support well. You can't change resolution within the one track, because it's AVC packetised video with only one set of SPS/PPS stored in the mp4 header.
It's likely a different track and it works fine here. Actually, it works with xvimagesink and glimagesink. With ximagesink, I see a green frame and few assertions, with cluttersink (hence Totem), it fails. But in general, the answer is that it works inside playbin, there might be issues with one of videoscale/videoconvert/ximagesink and there is definatly issues with cluttersink (which is a maintained externally from the gst project, but in the same bugzilla though).
Actually, even outside playbin it works fine: gst-launch-1.0 -v filesrc location=sample.mp4 ! qtdemux ! avdec_h264 ! xvimagesink
The stream is a bit weird because it's an AVC packetised stream with codecdata, but it then inserts a new SPS/PPS into the data anyway.
This is not a valid mp4 file then, use MPEG-TS or such if you need to change resolution/etc. in the middle of the stream. With AVC format SPS/PPS are supposed to be in the caps and when they change the caps (from the demuxer in this case) should change, I don't think we need to make the parser try to parse the stream anyway just in case someone created a bogus stream.