GNOME Bugzilla – Bug 792568
rtph264depay not forwarding dynamic caps changes to downstream
Last modified: 2018-01-16 21:11:48 UTC
In my usecase i am receiving raw RTP streams (h264) that may be reset and whose resolution can change. I am experiencing decoding artifacts when resolution changes. To reproduce: 1) start receiver pipeline below 2) start Sender1 <-- decoding works 3) stop Sender 1 4) start Sender2 <-- decoding artifacts I am expecting decoding to resume properly with the new encoding parameters. Receiver: gst-launch-1.0 udpsrc port=1234 ! "application/x-rtp, media=(string)video" ! rtpjitterbuffer ! rtph264depay ! h264parse ! avdec_h264 ! xvimagesink sync=false Sender1: gst-launch-1.0 videotestsrc is-live=true ! "video/x-raw, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(string)I420" ! x264enc tune=zerolatency speed-preset=ultrafast key-int-max=30 ! rtph264pay ! udpsink host=127.0.0.1 port=1234 Sender2: gst-launch-1.0 videotestsrc is-live=true ! "video/x-raw, width=(int)512, height=(int)240, framerate=(fraction)30/1, format=(string)I420" ! x264enc tune=zerolatency speed-preset=ultrafast key-int-max=30 ! rtph264pay ! udpsink host=127.0.0.1 port=1234 I confirm that rtph264depay seems to detect the new SPS/PPS ("copy SPS 0 of length 27", ...), but the encoder does not seem to reconfigure because it starts spitting out errors. I have confirmed the problem to happen with both libav_h264dec and vaapih264dec 00:17.091376964 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:1054:gst_rtp_h264_depay_process:<rtph264depay0> receiving 2 bytes 0:00:17.091391393 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:1074:gst_rtp_h264_depay_process:<rtph264depay0> NRI 0, Type 9 0:00:17.091399966 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:872:gst_rtp_h264_depay_handle_nal:<rtph264depay0> handle NAL type 9 0:00:17.091403225 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:932:gst_rtp_h264_depay_handle_nal:<rtph264depay0> start 0, complete 1 0:00:17.091421304 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:941:gst_rtp_h264_depay_handle_nal:<rtph264depay0> adding NAL to picture adapter 0:00:17.091428742 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:1054:gst_rtp_h264_depay_process:<rtph264depay0> receiving 27 bytes 0:00:17.091432037 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:1074:gst_rtp_h264_depay_process:<rtph264depay0> NRI 3, Type 7 0:00:17.091435603 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:872:gst_rtp_h264_depay_handle_nal:<rtph264depay0> handle NAL type 7 0:00:17.091459157 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:1054:gst_rtp_h264_depay_process:<rtph264depay0> receiving 4 bytes 0:00:17.091462561 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:1074:gst_rtp_h264_depay_process:<rtph264depay0> NRI 3, Type 8 0:00:17.091480838 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:872:gst_rtp_h264_depay_handle_nal:<rtph264depay0> handle NAL type 8 0:00:17.091502244 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:1054:gst_rtp_h264_depay_process:<rtph264depay0> receiving 690 bytes 0:00:17.091505779 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:1074:gst_rtp_h264_depay_process:<rtph264depay0> NRI 0, Type 6 0:00:17.091509999 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:872:gst_rtp_h264_depay_handle_nal:<rtph264depay0> handle NAL type 6 0:00:17.091530986 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:386:gst_rtp_h264_set_src_caps:<rtph264depay0> copy SPS 0 of length 27 0:00:17.091548759 20604 0x557c2a97fad0 DEBUG rtph264depay gstrtph264depay.c:402:gst_rtp_h264_set_src_caps:<rtph264depay0> copy PPS 0 of length 4 ... 0:00:17.092294852 20604 0x557c2a97fad0 ERROR libav :0:: top block unavailable for requested intra mode 0:00:17.092297768 20604 0x7faa400020a0 ERROR libav :0:: top block unavailable for requested intra mode 0:00:17.092307787 20604 0x7faa3c01be30 ERROR libav :0:: top block unavailable for requested intra mode 0:00:17.092338658 20604 0x7faa3c01be30 ERROR libav :0:: error while decoding MB 28 6 0:00:17.092305385 20604 0x557c2a97fad0 ERROR libav :0:: error while decoding MB 20 0
Is rtph264depay outputting avc or bytestream in your case? Did you confirm that it indeed does not forward the new SPS/PPS (in case of avc this would mean that the caps are changing!). The title of this bug sounds like it, but your comment sounds like you don't really know yet.
If it is outputting AVC, which I think it should in this case, then it's a duplicate of bug #790000 with some patches and the later comments explain what is left to be done.
Yes, i think it is AVC, sorry for the wrong title. I am closing this as invalid. @Sebastian, i'd be happy to learn how to debug whether the new SPS/PPS is forwarded.
*** This bug has been marked as a duplicate of bug 790000 ***
For AVC there would be new caps with a different codec_data