Are you saying this only happens within gst-rtsp-server but not in a plain gst-launch command line or why are you reporting it against gst-rtsp-server?

Similar pipelines with v4lsink or filesink work. I assume they provide the downstream buffer pool. I have only seen the problem up until now when using gst-rtsp-server. If I replace the v4l2h264enc by openh264enc without using dma buffers upstream I can stream, all be it with a lot of dropped frams. In this case I assume (but haven't checked) that openh264enc implements the decide_allocation and instantiates its own buffer pool.

(In reply to Keith Thornton from comment #0)
> using gst-rtsp-server with v4l2h264enc, the allocation query finds no
> element with a decide_allocation function so no downstream buffers are
> allocated.

The encoder does not run an allocation query. Encoded data is always pushed using the v4l2 driver allocated buffers, dmabuf by default. Why does it matter ? Do you actually have an error or something or is this just conceptually not what you expected ?

> example 
> ./test-launch "( v4l2src min-buffers=10

There is no min-buffers property in upstream v4l2src code. Can you explain which modifications you are running with ?

> device=/dev/v4l/by-path/platform-captu
> re-subsystem-video-index4
> video/x-raw,format=NV12,width=1920,height=1080,framera

This pipeline is unlinked, you need a ! between v4l2src and the caps filter.

> te=60000/1001 ! v4l2h264enc ! video/x-h264,profile=baseline ! rtph264pay
> name=pay0 pt=96 )"

(In reply to Keith Thornton from comment #2)
> Similar pipelines with v4lsink or filesink work. I assume they provide the
> downstream buffer pool. I have only seen the problem up until now when using
> gst-rtsp-server. If I replace the v4l2h264enc by openh264enc without using
> dma buffers upstream I can stream, all be it with a lot of dropped frams. In
> this case I assume (but haven't checked) that openh264enc implements the
> decide_allocation and instantiates its own buffer pool.

yes, im am experiencing an error, see below
the missing ! was a paste error
./test-launch "( v4l2src min-buffers=10 device=/dev/v4l/by-path/platform-captu
re-subsystem-video-index4 io-mode=4 ! video/x-raw,format=NV12,width=1920,height=
1080,framerate=60000/1001 ! v4l2h264enc output-io-mode=5 output-min-buffers=6 ca
pture-min-buffers=12 ! video/x-h264,profile=baseline ! queue max-size-buffers=3,
leaky=1 ! rtph264pay name=pay0 pt=96 )"

0:00:07.640716668   654 0x74f10320 DEBUG           videoencoder gstvideoencoder.c:1763:gst_video_encoder_negotiate_default:<v4l2h264enc0> didn't get downstream ALLOCATION hints
0:00:07.642418668   654 0x74f10320 DEBUG           videoencoder gstvideoencoder.c:1770:gst_video_encoder_negotiate_default:<v4l2h264enc0> ALLOCATION (1) params: allocation query: 0x73b03580, GstQueryAllocation, caps=(GstCaps)"video/x-h264\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)au\,\ profile\=\(string\)baseline\,\ level\=\(string\)4\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)60000/1001\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)bt709", need-pool=(boolean)true, allocator=(GArray)NULL, pool=(GArray)NULL, meta=(GArray)NULL;
0:00:07.839548335   654 0x74f10320 DEBUG           v4l2videoenc gstv4l2videoenc.c:352:gst_v4l2_video_enc_set_format:<v4l2h264enc0> output caps: video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)60000/1001, interlace-mode=(string)progressive, colorimetry=(string)bt709
0:00:07.840003001   654 0x74f10320 DEBUG           videoencoder gstvideoencoder.c:894:gst_video_encoder_sink_query:<v4l2h264enc0> received query 35846, allocation
0:00:07.840139335   654 0x74f10320 DEBUG           v4l2videoenc gstv4l2videoenc.c:859:gst_v4l2_video_enc_propose_allocation:<v4l2h264enc0> called
0:00:08.064016335   654 0x74f10320 DEBUG           videoencoder gstvideoencoder.c:1206:gst_video_encoder_sink_event:<v4l2h264enc0> received event 17934, segment
0:00:08.064191668   654 0x74f10320 DEBUG           videoencoder gstvideoencoder.c:1063:gst_video_encoder_sink_event_default:<v4l2h264enc0> segment time segment start=0:00:00.000000000, offset=0:00:00.000000000, stop=99:99:99.999999999, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0:00:00.000000000, base=0:00:00.000000000, position 0:00:00.000000000, duration 99:99:99.999999999
0:00:08.064498001   654 0x74f10320 LOG             videoencoder gstvideoencoder.c:1455:gst_video_encoder_chain:<v4l2h264enc0> received buffer of size 3110400 with PTS 0:00:01.099792114, DTS 99:99:99.999999999, duration 0:00:00.016683333
0:00:08.064711001   654 0x74f10320 LOG             videoencoder gstvideoencoder.c:1542:gst_video_encoder_chain:<v4l2h264enc0> passing frame pfn 0 to subclass
0:00:08.064829335   654 0x74f10320 DEBUG           v4l2videoenc gstv4l2videoenc.c:714:gst_v4l2_video_enc_handle_frame:<v4l2h264enc0> Handling frame 0
0:00:08.065129001   654 0x74f10320 WARN            v4l2videoenc gstv4l2videoenc.c:782:gst_v4l2_video_enc_handle_frame:<v4l2h264enc0> error: Failed to allocate required memory.
0:00:08.065250001   654 0x74f10320 WARN            v4l2videoenc gstv4l2videoenc.c:782:gst_v4l2_video_enc_handle_frame:<v4l2h264enc0> error: Buffer pool activation failed
0:00:08.066023001   654 0x74f10320 DEBUG           videoencoder gstvideoencoder.c:1206:gst_video_encoder_sink_event:<v4l2h264enc0> received event 28174, eos
0:00:08.071124335   654  0x1f46290 DEBUG           v4l2videoenc gstv4l2videoenc.c:200:gst_v4l2_video_enc_stop:<v4l2h264enc0> Stopping
0:00:08.083856001   654  0x1f46290 DEBUG           v4l2videoenc gstv4l2videoenc.c:224:gst_v4l2_video_enc_stop:<v4l2h264enc0> Stopped
0:00:08.085544335   654  0x1f46290 DEBUG           v4l2videoenc gstv4l2videoenc.c:171:gst_v4l2_video_enc_close:<v4l2h264enc0> Closing

I'm not aware of these patches adding properties like "output-min-buffers" and "capture-min-buffers". Please document the changes you apply to GStreamer, otherwise it's really hard to help.

In this context, you should also provide a trace with "GST_DEBUG="v4l2*:7" so we wee what is happening on V4L2 side. So far we only see that the decoder refused to start, apparently due to allocation failure, but there was a error message fix recently for this specific error, so I don't know if this is exact or not.

About traces, please, attach them instead of pasting them, they are pretty hard to read due to the reflow.

you are correct, we have some not mainlined test-patches either from my company or from a supporting company. I will try to get hold of these patches or run the same test with an unpached version. The same problems occur without using these additional parameters. A v4l2*:7 listing makes the server so slow that the client dismantles the connection due to a timeout :-(

Michael Tretter traced this problem and has determined the following
gst-rtsp-server links the RtpBin after the pipeline has been set to running. This leads to a reconfigure event being sent to the v4l2videoenc. The element tries a re-format but this is refused because the element (the driver) is in streaming mode. In our case, the caps are identical to the previously negotiated caps so it could continue without re-configuration. If a reconfiguration were necessary, the encoder would need to be stopped and then re-started.

Which version of GStreamer is this ? The reconfiguration code has been added in 1.14, prior to that it would have failed. Also, are you going to provide a patch that detect that caps haven't changed and thus reallocation is not needed ?

we are running 1.14.0
We are working on a patch but it is not yet correct.

Created attachment 372813 [details] [review]
0001-v4l2videoenc-Hack-reconfiguration-while-encoder-is-s.patch

Review of attachment 372813 [details] [review]:

Renegotiation should work, it's just a bug. All bits are in place, no need for hacks.

I quite agrree, it should work but with test-launch it didn't :-(

(In reply to Nicolas Dufresne (ndufresne) from comment #11)
> Review of attachment 372813 [details] [review] [review]:
> 
> Renegotiation should work, it's just a bug. All bits are in place, no need
> for hacks.

Yes, there shouldn't be the need for the hack and it wasn't meant for inclusion into master. The patch only helps to track down the actual bug.

During renegotiation the encoder element calls set_format for capture. Since v4l2 is currently in streaming mode, set_format fails to change the format and the entire renegotiation fails.

I think during renegotiation we should (1) stop streaming, (2) set the format and (3) start streaming with the new format. This is currently not implemented, but this should fix renegotiation for the common case.

The patch that I submitted assumes that the caps are the same as prior to the renegotiation event (there is no check if this is actually true). If we detect this condition, we can skip the set_format and just continue the renegotiation. However, I wasn't able to detect this case in v4l2videoenc, and I don't know if this should be handled in the videoencoder.

Ok, there is two form of renegotiation, firs one, there is new caps on the sink pad, that's the one covered. You'll see the two calls to v4l2_object_stop() in there, which do STREAMOFF.

The second one is the renegotiation event on the srcpad (caused by linking downstream). I think this one is not currently covered and am guessing that this is what happens. There is no obligation to change the caps though on this one, the renegotiation event is a suggestion. Worst, changing the caps would flush the encoder, hence produce a new IDR for nearly no reason.

Can anyone tell me what other encoders do ?

*** Bug 794400 has been marked as a duplicate of this bug. ***

Review of attachment 372813 [details] [review]:

::: sys/v4l2/gstv4l2videoenc.c
@@ +824,3 @@
+   * the caps, because the videoencoder just set encoder->srcpad caps to
+   * state->caps. */
+  if (GST_V4L2_IS_ACTIVE (self->v4l2capture)) {

It's a bit late, we have already done bunch of illegal calls in _negotiate(). decide_allocation is called by the base class implementation of negotiate(). If negotiate() is called with an self->input_state set, it means we are not being called from _set_format(). I'm surprise running decide_allocation worked, what I would do is simply return TRUE in negotiate() if the reason we negotiate is not a new input format. Replying to reconfigure event is optional anyway. I'll test this tomorrow and report.

Created attachment 372873 [details] [review]
v4l2videoenc: Only renegotiate with upstream

When the decoder get linked further, it will receive a renegotiation
event from downstream. This case is not supported and should be ignored.

This fixes issues when this encoder is used inside an GstRtspServer
pipeline.

This fixes the issue for me, let me know how it goes for you.

Attachment 372873 [details] pushed as 047dc61 - v4l2videoenc: Only renegotiate with upstream

The patch works for me. We (you or me?) can close the bug.
Thank you.

It's already marked resolved fixed, I've backported to 1.14 branch, so rtsp server on imx6 should work in 1.14.2.