GNOME Bugzilla – Bug 669509
h264parse/aacparse: caps renegotiation fails due to baseparse calling gst_pad_use_fixed_caps
Last modified: 2018-11-03 13:10:43 UTC
Created attachment 206935 [details] [review] Patch to allow downstream caps renegotiation I have: vsrc ! videorate ! omxh264enc ! h264parse ! fakesink Video source has variable frame rate and videorate force-fps is set dynamically on the running pipeline. This triggers downstream caps negotiation that fails in h264parse. * upstream calls gst_pad_get_allowed_caps on h264parse:sink * gst_h264_parse_get_caps is called to determine allowed caps * gst_h264_parse_get_caps calls gst_pad_get_allowed_caps on h264parse:src * gst_pad_get_allowed_caps on h264parse:src return the current caps * current caps and new caps differ in frame rate and new caps are rejected The reason for failure: baseparse is calling gst_pad_use_fixed_caps on h264parse:src, so the current caps is returned. From docs "Use this function on a pad that, once gst_pad_set_caps() has been called on it, cannot be renegotiated to something else." This is not appropriate for h264 as it can change its properties mid stream (new SPS/PPS). For example, frame-rate may be variable, for example in video conferencing. Example: gst_pad_get_allowed_caps (h264parse:src) returns video/x-h264,width=640,height=480,framerate=1/1,profile=baseline,level=1.1, parsed=true,stream-format=byte-stream,alignment=au gst_pad_get_allowed_caps (h264parse:sink) returns video/x-h264,width=640,height=480,framerate=1/1,profile=baseline,level=1.1 Encoder (gst-omx / basevideoencoder) think downstream can only handle framerate=1/1 and rejects new caps with framerate=2/1 even though downstream actually can handle it.
Instead of this it would be better to have a vfunc for getcaps() on the srcpad too. This would default to the fixed caps function and in h264parse it would be re-implemented to properly return the caps we can create.
I'm 99.99% sure this is fixed in 1.x. Jonas, do you still get the issue in 1.x ?
A similar bug is reproducing on GStreamer 1.4.5 and 1.6.3, and probably all others. The issue boils down to: - aacparse used for adaptive streaming AAC - the first segment uses sample rate 22050 (caps: audio/mpeg, framed=(boolean)true, mpegversion=(int)4, level=(string)1, base-profile=(string)lc, profile=(string)lc, rate=(int)22050, channels=(int)2, stream-format=(string)adts) - the second segment uses rate 44100 - in gst_aac_parse_set_src_caps(), gst_pad_get_allowed_caps (GST_BASE_PARSE (aacparse)->srcpad) returns the old negotiated caps, with sample rate 22050, - new caps can not intersect, we enter a special branch for handling this problem, - in that branch, the aacparse simply changes stream format from adts to raw, and retries, - needless to say, due to different rate, gst_caps_can_intersect (src_caps, allowed) fails - so aacparse does not even set "codec_data" caps. Simply clearing GST_PAD_FLAG_FIXED_CAPS flag does the job - caps are negotiated correctly and playback works. Now, the playback works on x86, only because it (due to the same bug) changes caps *from* raw into adts, so no codec_data is required. See the reproduction on x86: Anyway, the testcase is: export GST_DEBUG=aacparse:5 gst-launch-1.0 playbin uri=http://yt-dash-mse-test.commondatastorage.googleapis.com/media/car-20120827-manifest.mpd The logs: Setting pipeline to PAUSED ... Pipeline is PREROLLING ... 0:00:00.330564508 29657 0x7ff0dc14d280 DEBUG aacparse gstaacparse.c:145:gst_aac_parse_init: initialized 0:00:00.330778462 29657 0x7ff0dc14d280 DEBUG aacparse gstaacparse.c:1415:gst_aac_parse_start: start 0:00:00.728723676 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:291:gst_aac_parse_sink_setcaps:<aacparse0> setcaps: audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)1, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1388, rate=(int)22050, channels=(int)1 0:00:00.728763502 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:325:gst_aac_parse_sink_setcaps: codec_data: object_type=2, sample_rate=22050, channels=1, samples=1024 0:00:00.728778072 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:172:gst_aac_parse_set_src_caps:<aacparse0> sink caps: audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)1, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1388, rate=(int)22050, channels=(int)1 0:00:00.729981179 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:256:gst_aac_parse_set_src_caps:<aacparse0> setting src caps: audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)1, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1388, rate=(int)22050, channels=(int)1 Redistribute latency... Prerolled, waiting for buffering to finish... Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstPulseSinkClock Redistribute latency... 0:00:10.519105212 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:291:gst_aac_parse_sink_setcaps:<aacparse0> setcaps: audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2 0:00:10.519141905 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:325:gst_aac_parse_sink_setcaps: codec_data: object_type=2, sample_rate=44100, channels=2, samples=1024 0:00:10.519157027 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:172:gst_aac_parse_set_src_caps:<aacparse0> sink caps: audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2 0:00:10.520443564 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:221:gst_aac_parse_set_src_caps:<aacparse0:src> Caps can not intersect 0:00:10.520480456 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:243:gst_aac_parse_set_src_caps:<aacparse0:src> Input is raw, trying ADTS 0:00:10.520494939 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:256:gst_aac_parse_set_src_caps:<aacparse0> setting src caps: audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)adts, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2 You can see how intitially the aac stream is configured to be: audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)1, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1388, rate=(int)22050, channels=(int)1 And when new caps come, new caps change the level, obviously codec_data, and rate, but then: 0:00:10.520443564 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:221:gst_aac_parse_set_src_caps:<aacparse0:src> Caps can not intersect 0:00:10.520480456 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:243:gst_aac_parse_set_src_caps:<aacparse0:src> Input is raw, trying ADTS 0:00:10.520494939 29657 0x7ff0c80035e0 DEBUG aacparse gstaacparse.c:256:gst_aac_parse_set_src_caps:<aacparse0> setting src caps: audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)adts, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2 Whoopsie, because these caps were not compatible with previous negotiated caps, and the pad works in fixed mode, aacparse changed stream-format into ADTS!
Why does the parser use fixed caps on the srcpad anyway? That doesn't seem right.
It might be a historical thing, from GStreamer 0.10.32: Author: Mark Nauwelaerts <mark.nauwelaerts@collabora.co.uk> 2009-10-28 12:00:08 Committer: Tim-Philipp Müller <tim.muller@collabora.co.uk> 2011-04-08 16:44:47 Parent: 9f5bb92e74fde1d8f0218c6019de15e6f175af5c (aacparse: Fix compilation warnings) Child: 899a6de75b00ed6f9bf65338f2f66787133f0f60 (baseparse: documentation fixes) Branches: 1.4.5-with-backports, master, remotes/origin/1.4.5-with-backports, remotes/origin/CES-2013-BCM, remotes/origin/RELEASE-0.10.35-bison-fix, remotes/origin/SDK-4.3-bison-fix, remotes/origin/master, remotes/origin/upstream-0.10 Follows: RELEASE-0.10.32 Precedes: RELEASE-0.10.33 baseparse: use_fixed_caps for src pad After all, stream is as-is, and there is little molding to downstream's taste that can be done. If subclass can and wants to do so, it can still override as such. Then, that line that sets fixed caps was propagated when baseparse was separated from aacparse, then when it was moved from audioparsers to baseparse, then into GStreamer 1.x.
I think generally the fixed caps don't make sense on any srcpad of a parser at least, or the parsers all need to work around the flag when querying downstream caps.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/60.