GNOME Bugzilla – Bug 672019
rtph264pay ! rtph264depay restricts input caps (regression)
Last modified: 2012-05-24 08:09:01 UTC
profile=(string)constrained-baseline is added to the sinkpad caps of rtph264pay if the srcpad has been linked to a peer pad that does not have profile-level-id in caps. Example: GST_DEBUG=2 gst-launch-0.10 filesrc location=h264_main_profile.mkv ! matroskademux ! rtph264pay ! rtph264depay ! fakesink Setting pipeline to PAUSED ... Pipeline is PREROLLING ... 0:00:00.017022666 26975 0xe95cf0 WARN matroskademux matroska-demux.c:4498:gst_matroska_demux_loop:<matroskademux0> error: stream stopped, reason not-linked ERROR: from element /GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0: GStreamer encountered a general stream error. Additional debug info: matroska-demux.c(4498): gst_matroska_demux_loop (): /GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0: stream stopped, reason not-linked ERROR: pipeline doesn't want to preroll. Setting pipeline to NULL ... Freeing pipeline ... This patch would make it work: diff --git a/gst/rtp/gstrtph264pay.c b/gst/rtp/gstrtph264pay.c index 87c9b7e..80a5797 100644 --- a/gst/rtp/gstrtph264pay.c +++ b/gst/rtp/gstrtph264pay.c @@ -367,11 +367,6 @@ gst_rtp_h264_pay_getcaps (GstBaseRTPPayload * payload, GstP gst_structure_set (new_s, "profile", G_TYPE_STRING, "constrained-baseline", NULL); } - } else { - /* No profile-level-id also means baseline */ - - gst_structure_set (new_s, - "profile", G_TYPE_STRING, "constrained-baseline", NULL); } gst_caps_merge_structure (caps, new_s);
Right, so what it's doing is actually not quite right IMHO. What it should be doing if there are no profile restrictions downstream is return caps with profile=constrained-baseline in the first structure, and then unrestricted caps in the second structure. That way, an encoder will/should still pick the first, but one can still feed anything.
I'm afraid that this will might still yield the wrong thing.. As the encoder could still send something that the other side can't accept. I guess that in your usecase, you should have a capsfilter after the payloader with something like this where you state which profiles are acceptable (as I'm pretty sure you don't want to send the Extended or MultiView profiles): "application/x-rtp, encoding-name=h264, profile-level-id=423200; # baseline level 5.0 application/x-rtp, encoding-name=h264, profile-level-id=4D3200; # main, level 5.0 application/x-rtp, encoding-name=h264, profile-level-id=643200" # high, level 5.0 I agree this is a bit annoying, but I can't figure out a better way to do this as I understand that you're actually getting your media from a file. As for the level, just put a really high value, I hope that in 0.11, we can put the actual content as a declaratif profile-level-id in the STREAM_CONFIG sticky event.
Well sure, you can always construct stuff that won't work. That's just how it is with RTP without external signalling, where you just send packets into a hole and hope and pray. If there is indeed an encoder (and no particular profile/level is configured for output), it should negotiate based on the order of the caps structures returned by the get_caps(), so in this case to something baseline-ish. If it doesn't do that, the encoder is either broken or configured to do something else. I don't think we can just suddenly require people to add capsfilters after the payloader now where it wasn't required before, and it seems a bit broken in principle too. Perhaps I'm still missing something, but I can't say I see why my suggestion isn't workable and wouldn't solve the problem for the most part in practice.
*** Bug 676028 has been marked as a duplicate of this bug. ***
We got performance problems when adding a capsfilter and the reason for that is that the capsfilter does not support GstBufferList. So I think we need to either fix this bug the way Tim suggests or add support for buffer lists in capsfilter (or basetransform).
Created attachment 214554 [details] [review] Add unrestricted caps Here's a patch for adding unrestricted caps for the case when profile-level-id is missing. The case when profile-level-id is invalid will still set restricted caps.
(In reply to comment #6) > Created an attachment (id=214554) [details] [review] > Add unrestricted caps > > Here's a patch for adding unrestricted caps for the case when profile-level-id > is missing. The case when profile-level-id is invalid will still set restricted > caps. It is line with the suggestion in Comment #1, and fwiw is fine by me in that regard. The only nitpick is that one might want to add the unrestricted structure all the way at the end (so outside the loop), to somehow really make it the last possible resort (to address some other concerns stated earlier).
Created attachment 214602 [details] [review] Add unrestricted caps as last structure Updated patch according to Comment #7
Shoudl we not patch rtph264depay to offer the various profiles instead? (ie, to proxy the profiles from the decoder?)
The pipeline I'm having trouble with doesn't really contain a depayloader. I'm linking the payloader to gstrtpbin.
The you should probably have a capsfilter with the caps that you get from the receiver.
I already tried the capsfilter approach, it works but performance is really bad because the capsfilter does not let buffer lists through. It will "convert" the buffer lists created by rtph264pay to normal buffers.
Good point about buffer lists, I guess 0.11 fixes that.. Ok, let's add a unfiltered application/x-rtp at the end and we can fix it properly in 0.11.
commit ad38d163aa8c7569d584465134b9f6a01052f468 Author: Jonas Holmberg <jonashg@axis.com> Date: Mon May 21 12:17:35 2012 +0200 rtph264pay: Add unrestricted caps If there are no profile restrictions downstream, return caps with profile=constrained-baseline in the first structure and append unrestricted caps as the last structure. Fixes bug #672019
*** Bug 676715 has been marked as a duplicate of this bug. ***