GNOME Bugzilla – Bug 607452
Failure to sync on rtpmp4vpay stream; sender;receiver mismatch
Last modified: 2010-05-04 09:33:10 UTC
I am standardising to an RTP stream from an proprietary communcation format. The barcosoapsrc is a wrapper around a SOAP interface that negociates the udp address of an RTP stream. GST_DEBUG=*barco*:5 gst-launch barcosoapsrc location=pelco+soap://10.4.0.36:49152 ! rtpmp4vdepay ! mpeg4videoparse ! queue ! decodebin ! autovideosink This gets the stream and decodes is as expected. When taking this RTP stream, depayloading it as for decoding; but then payloading it again and sending it out over the network; the resulting payloaded stream that is recieved does not decode. sending: GST_DEBUG=*barco*:5 gst-launch barcosoapsrc location=pelco+soap://10.4.0.36:49152 ! rtpmp4vdepay ! mpeg4videoparse ! queue ! rtpmp4vpay ! udpsink host=226.226.226.22 port=2266 receiving: gst-launch udpsrc uri=udp://226.226.226.22:2266 caps="application/x-rtp, clock-rate=(int)90000" ! queue ! rtpmp4vdepay ! queue ! decodebin ! autovideosink veryfing with: gst-launch udpsrc uri=udp://226.226.226.22:2266 caps="application/x-rtp, clock-rate=(int)90000" ! queue ! rtpmp4vdepay ! fakesink dump=1 and gst-launch udpsrc uri=udp://226.226.226.22:2266 caps="application/x-rtp, clock-rate=(int)90000" ! queue ! rtpmp4vdepay ! mpeg4videoparse ! fakesink dump=1 seems to indicate that mpeg4videparse is blocking data (while it has been parsed on the recieve side; directly from the encoder, see first command line). However, when starting the reciever *before* the sender; data is being received and decoded (with mpeg4videoparse). Looks similar to the NAL 7/8 packets in H264.
Verified by listening to the udp socket that is being negociated by the soap calls too, instead of using the module. 1. use soap wrapper to get udp mc address 2. listen to mc address using udpsrc (mc address seems to be identical between calls). Same behaviour; mpeg4videoparse only passes data if it was running *before* the sender got active.
Starting the stream in this sequence (receiver before transmitter) decodes the stream as expected: 1. receiver/decoder gst-launch udpsrc uri=udp://226.226.226.22:2266 caps="application/x-rtp, clock-rate=(int)90000" ! queue ! rtpmp4vdepay ! mpeg4videoparse ! queue ! decodebin ! autovideosink 2. listen to udp stream negociated by number 3 gst-launch udpsrc uri=udp://236.4.0.36:9010 caps="application/x-rtp, clock-rate=(int)90000" ! rtpmp4vdepay ! mpeg4videoparse ! queue ! rtpmp4vpay send-config=True ! udpsink host=226.226.226.22 port=2266 3. init soap/udp stream and fakesink. gst-launch barcosoapsrc location=pelco+soap://10.4.0.36:49152 ! fakesink
mpeg4videoparse opens the stream when it finds a VOS packet (blocks it before) and allows rtpmp4vpay be configured. Seems suspiciously identical to H264 where we ended up multiplexing NAL 7/8 sps/pps again in the rtph264pay; only here it is in the VOS header in MPEG4 video; looking if it is allowed to apply the same trick as in H264 by slightly modifying rtpmp4vpay and where it should be inserted in the stream. GST_DEBUG_NO_COLOR=1 GST_DEBUG=*mpeg4videoparse*:5 gst-launch udpsrc uri=udp://236.4.0.36:9010 caps="application/x-rtp, clock-rate=(int)90000" ! rtpmp4vdepay ! mpeg4videoparse ! queue ! rtpmp4vpay send-config=True ! udpsink host=226.226.226.22 port=2266 0:00:03.670503026 16952 0x125cf30 LOG mpeg4videoparse mpeg4videoparse.c:553:gst_mpeg4vparse_drain:<mpeg4vparse0> found VOS start marker at 0 0:00:03.670515173 16952 0x125cf30 DEBUG mpeg4videoparse mpeg4videoparse.c:287:gst_mpeg4vparse_handle_vo:<mpeg4vparse0> time increment resolution 25025 0:00:03.670523020 16952 0x125cf30 DEBUG mpeg4videoparse mpeg4videoparse.c:303:gst_mpeg4vparse_handle_vo:<mpeg4vparse0> fixed time increment 1001 0:00:03.670562292 16952 0x125cf30 DEBUG mpeg4videoparse mpeg4videoparse.c:116:gst_mpeg4vparse_set_new_caps:<mpeg4vparse0> setting downstream caps to video/mpeg, mpegversion=(int)4, systemstream=(boolean)false, parsed=(boolean)true, profile-level-id=(string)3, codec_data=(buffer)000001b003000001b509000001000000012000c888b0e0e0fa62c089028307, framerate=(fraction)25/1, pixel-aspect-ratio=(fraction)1/1, width=(int)704, height=(int)576 0:00:03.670624600 16952 0x125cf30 LOG mpeg4videoparse mpeg4videoparse.c:544:gst_mpeg4vparse_drain:<mpeg4vparse0> found VOP start marker at 31 0:00:03.670709882 16952 0x125cf30 LOG mpeg4videoparse mpeg4videoparse.c:600:gst_mpeg4vparse_drain:<mpeg4vparse0> found VOP end marker at 23288 GST_DEBUG_NO_COLOR=1 GST_DEBUG=*rtpmp4vpay*:5 gst-launch udpsrc uri=udp://236.4.0.36:9010 caps="application/x-rtp, clock-rate=(int)90000" ! rtpmp4vdepay ! mpeg4videoparse ! queue ! rtpmp4vpay send-config=True ! udpsink host=226.226.226.22 port=2266 0:00:03.378564183 16904 0x1e1cd90 LOG rtpmp4vpay gstrtpmp4vpay.c:244:gst_rtp_mp4v_pay_setcaps:<rtpmp4vpay0> got codec_data 0:00:03.378590917 16904 0x1e1cd90 LOG rtpmp4vpay gstrtpmp4vpay.c:260:gst_rtp_mp4v_pay_setcaps:<rtpmp4vpay0> configuring codec_data, profile 3 0:00:03.378663218 16904 0x1e1cd90 DEBUG rtpmp4vpay gstrtpmp4vpay.c:395:gst_rtp_mp4v_pay_depay_data:<rtpmp4vpay0> start code 0x000001b00:00:03.378673393 16904 0x1e1cd90 DEBUG rtpmp4vpay gstrtpmp4vpay.c:410:gst_rtp_mp4v_pay_depay_data:<rtpmp4vpay0> VOS profile 0x00000003
Created attachment 151844 [details] [review] add config-interval parameter to re-insert config in stream this re-inserts the config in the rtp stream: there are encoders that send the config only once and when a stream is transmitted onto a different socket with gstreamer; the MPEG4 cannot be decoded/interpreted by pipelins picking in on the stream afterwards.
Created attachment 152472 [details] [review] add config-interval parameter to re-insert config in stream added check if we have a valid config: we should not merge append the data to the config buffer if the timer expires if there is not valid config stored yet (NULL ptr). rarely noticed during startup.
Created attachment 159239 [details] [review] add config-interval parameter to re-insert config in stream Modified/fixed version of previous patch (note that no config will be inserted if send-config is not TRUE in the first place, or it would otherwise be somewhat contradictory).
Pushed a slightly modified version of attached patch. commit 220f865f77e732d6f3496506a27bb2f2ad5f7853 Author: Mark Nauwelaerts <mark.nauwelaerts@collabora.co.uk> Date: Tue May 4 11:13:45 2010 +0200 rtpmp4vpay: add config-interval parameter to re-insert config in stream Add a new config-interval property to instruct the payloader to insert config (VOSH, VOS, etc) at periodic intervals in the stream (when a GOP or VOP-I is encountered). Based on patch by <marc.leeman at gmail.com> Fixes #607452.