GNOME Bugzilla – Bug 794400
rtspclientsink: Issues with Wowza when using v4l2h264enc on Dragonboard 410c
Last modified: 2018-06-28 21:02:27 UTC
It seems that when using the v4l2h264enc element with the rtspclientsink has a few issues when using the Dragonboard 410c's video encoder. Basically I cannot seem to get the board to start delivering a stream to a remote server running on Wowza's Video Cloud service using the following pipeline: gst-launch-1.0 -v v4l2src device=/dev/video3 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! v4l2h264enc extra-controls="controls,h264_profile=4,video_bitrate=20000 00;" ! video/x-h264,profile=main,level=4,stream-format=byte-stream,alignment=au ! rtspclientsink location=rtsp://<snipped>.entrypoint.cloud.wowza.com/app-<snipped>:1935 Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Progress: (open) Opening Stream Progress: (connect) Connecting to rtsp://<snipped>.entrypoint.cloud.wowza.com/app-<snipped>:1935 /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1, pixel-aspect-ratio=(fraction)1/1 /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1, pixel-aspect-ratio=(fraction)1/1 /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)main, level=(string)1, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1 Redistribute latency... /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1, pixel-aspect-ratio=(fraction)1/1 /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1, pixel-aspect-ratio=(fraction)1/1 ERROR: from element /GstPipeline:pipeline0/GstRTSPClientSink:rtspclientsink0: Could not open resource for reading and writing. Additional debug info: ../../../gst-rtsp-server-1.12.2/gst/rtsp-sink/gstrtspclientsink.c(2924): gst_rtsp_client_sink_connect_to_server (): /GstPipeline:pipeline0/GstRTSPClientSink:rtspclientsink0: Failed to connect. (Generic error) ERROR: pipeline doesn't want to preroll. Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ... Recommended reproduction steps: 1. Sign up for a Wowza Video Cloud trial - https://www.wowza.com/pricing/cloud-free-trial 2. Bring up a Dragonboard 410c and camera (I use an OV5640 on a mezzanine) running the kernel mentioned in this post (or the current 18.01 release) 3. Install GStreamer 1.12.2 and gstrtspserver 4. Set up a cloud video endpoint - https://www.wowza.com/docs/getting-started-with-wowza-streaming-cloud (I used the other RTSP option to setup streaming) 5. Attempt streaming using the pipeline referenced above Some information about my setup: Kernel Version: 4.14.15 (Link to repo: https://github.com/RobGries/96b_4.14_kernel) Device: Dragonboard 410c GStreamer-1.0 version: GStreamer Core Library version 1.12.2
I went ahead and captured some more information using the GST_DEBUG=2 flag, also I mistakenly had the endpoint shutdown on the cloud side. I have now resolved that, and it definitely seems like I cannot get the encoder to properly map a buffer for the frames incoming from the v4l2h264enc element: GST_DEBUG=2 gst-launch-1.0 -v v4l2src device=/dev/video3 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! v4l2h264enc extra-controls="controls,h264_profile=4,video_bitrate=2000000;" ! video/x-h264,profile=main,level=4,stream-format=byte-stream,alignment=au ! mpegtsmux name=mux ! rtspclientsink location= rtsp://<snipped>.entrypoint.cloud.wowza.com/<snipped>:1935 Setting pipeline to PAUSED ... 0:00:00.626412760 9321 0x23951430 WARN v4l2 gstv4l2object.c:1860:gst_v4l2_object_add_aspect_ratio:<v4l2h264enc0> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument 0:00:00.626700890 9321 0x23951430 WARN v4l2 gstv4l2object.c:1860:gst_v4l2_object_add_aspect_ratio:<v4l2h264enc0> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument 0:00:00.626881884 9321 0x23951430 WARN v4l2 gstv4l2object.c:1860:gst_v4l2_object_add_aspect_ratio:<v4l2h264enc0> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument 0:00:00.627037720 9321 0x23951430 WARN v4l2 gstv4l2object.c:1860:gst_v4l2_object_add_aspect_ratio:<v4l2h264enc0> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument Pipeline is live and does not need PREROLL ... Progress: (open) Opening Stream Progress: (connect) Connecting to rtsp://<snipped>.entrypoint.cloud.wowza.com/<snipped>:1935 /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1, pixel-aspect-ratio=(fraction)1/1 /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1, pixel-aspect-ratio=(fraction)1/1 0:00:01.138369121 9321 0x2394fd40 WARN GST_PADS gstpad.c:4109:gst_pad_peer_query:<v4l2h264enc0:src> could not send sticky events /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)main, level=(string)1, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1 Redistribute latency... 0:00:01.140412912 9321 0x2394fd40 WARN v4l2bufferpool gstv4l2bufferpool.c:754:gst_v4l2_buffer_pool_start:<v4l2h264enc0:pool:src> Uncertain or not enough buffers, enabling copy threshold /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1, pixel-aspect-ratio=(fraction)1/1 /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1, pixel-aspect-ratio=(fraction)1/1 Progress: (open) Retrieving server options Progress: (open) Opened Stream Setting pipeline to PLAYING ... New clock: GstSystemClock /GstPipeline:pipeline0/GstRTSPClientSink:rtspclientsink0/GstBin:rtspbin/GstRtpBin:rtpbin0: latency = 2000 /GstPipeline:pipeline0/GstRTSPClientSink:rtspclientsink0/GstBin:rtspbin/GstRtpBin:rtpbin0: ntp-time-source = NTP time based on realtime clock 0:00:01.435180408 9321 0x2394fd40 WARN v4l2bufferpool gstv4l2bufferpool.c:754:gst_v4l2_buffer_pool_start:<v4l2h264enc0:pool:sink> Uncertain or not enough buffers, enabling copy threshold 0:00:01.472391047 9321 0xffffa4002940 WARN v4l2bufferpool gstv4l2bufferpool.c:1207:gst_v4l2_buffer_pool_dqbuf:<v4l2h264enc0:pool:src> Driver should never set v4l2_buffer.field to ANY 0:00:01.476860563 9321 0x2394fd40 WARN basesrc gstbasesrc.c:2939:gst_base_src_loop:<v4l2src0> error: Internal data stream error. 0:00:01.477054786 9321 0x2394fd40 WARN basesrc gstbasesrc.c:2939:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-negotiated (-4) 0:00:01.478028659 9321 0x2394fd40 WARN mpegtsmux mpegtsmux.c:868:mpegtsmux_create_streams:<mux> error: Could not create handler for stream ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error. Additional debug info: ../../../../gstreamer-1.12.2/libs/gst/base/gstbasesrc.c(2939): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: streaming stopped, reason not-negotiated (-4) Execution ended after 0:00:00.050197043 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ...
Have you tried if it works with gst-rtsp-server ?
Is it me or your 1.12.2 build is patched ? Also, did you try with 1.14 rc2 ?
@Tim-Philipp Müller: I modified the test code located at https://github.com/GStreamer/gst-rtsp-server/blob/master/examples/test-video.c and it appears to be working: https://gist.github.com/RobGries/72d4140ff8ff0418732637f2cfb1a166 @Nicolas Dufresne: Yes, my build is patched using the following patches: https://source.codeaurora.org/quic/imm/imm/meta-backports/tree/core/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good?h=linaro/morty I have not tried testing with 1.14 rc2 just yet, but I'll get started on building that version now and I'll report back with my findings.
Also, I'm wondering if rtspclientsink could just not have enough information in the caps to produce a valid SDP. Can you try adding h264parse after v4l2h264enc ?
@Nicolas Dufresne: I was able to get 1.13 to compile via a new Yocto recipe that I've dropped into my build, but this does not seem to make a difference, even with the added h264parse element between v4l2h264enc and the rtspclient sink element. I'm still working on getting the 1.14 rc2 version to build via bitbake and will continue on it and report back when I make any meaningful progress. My new log is below: GST_DEBUG=2 gst-launch-1.0 -v v4l2src device=/dev/video3 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! v4l2h264enc extra-controls="controls,h264_profile=4,video_bitrate=2000000;" ! video/x-h264,profile=main,level=4,stream-format=byte-stream,alignment=au ! h264parse ! rtspclientsink location=rtsp://2113fb.entrypoint.cloud.wowza.com/app-c53a:1935 Setting pipeline to PAUSED ... 0:00:00.619713639 7612 0x27d86630 WARN v4l2 gstv4l2object.c:4071:gst_v4l2_object_probe_caps:<v4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument Pipeline is live and does not need PREROLL ... Progress: (open) Opening Stream Progress: (connect) Connecting to rtsp://2113fb.entrypoint.cloud.wowza.com/app-c53a:1935 /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1 /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1 0:00:01.116968372 7612 0x27d84450 WARN GST_PADS gstpad.c:4179:gst_pad_peer_query:<v4l2h264enc0:src> could not send sticky events /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)main, level=(string)1, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1 Redistribute latency... 0:00:01.119411476 7612 0x27d84450 WARN v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<v4l2h264enc0:pool:src> Uncertain or not enough buffers, enabling copy threshold /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1 /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:7:1 Progress: (open) Retrieving server options Progress: (open) Opened Stream Setting pipeline to PLAYING ... New clock: GstSystemClock Progress: (request) Sending RECORD request /GstPipeline:pipeline0/GstRTSPClientSink:rtspclientsink0/GstBin:rtspbin/GstRtpBin:rtpbin0: latency = 2000 /GstPipeline:pipeline0/GstRTSPClientSink:rtspclientsink0/GstBin:rtspbin/GstRtpBin:rtpbin0: ntp-time-source = NTP time based on realtime clock 0:00:01.376042406 7612 0x27d84450 WARN v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<v4l2h264enc0:pool:sink> Uncertain or not enough buffers, enabling copy threshold 0:00:01.408890051 7612 0xffff9c002f70 WARN v4l2bufferpool gstv4l2bufferpool.c:1181:gst_v4l2_buffer_pool_dqbuf:<v4l2h264enc0:pool:src> Driver should never set v4l2_buffer.field to ANY 0:00:01.412465564 7612 0x27d84450 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<v4l2src0> error: Internal data stream error. 0:00:01.412515721 7612 0x27d84450 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-negotiated (-4) ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error. Additional debug info: ../../../../git/libs/gst/base/gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: streaming stopped, reason not-negotiated (-4) Execution ended after 0:00:00.149464867 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ...
Thanks for taking the time to report this. This particular bug has already been reported into our bug tracking system, but please feel free to report any further bugs you find. *** This bug has been marked as a duplicate of bug 796525 ***