GNOME Bugzilla – Bug 726521
v4l2src fails to enumerate Raspberry Pi camera which specifies stepwise frame sizes
Last modified: 2014-07-01 19:09:24 UTC
The Raspberry Pi Foundation markets a small camera board for which a V4l2 driver now became available that is usable by, e.g., v4l2src. After an update that introduced so-called stepwise frame sizes [1], v4l2src cannot negotiate a camera mode anymore, which worked before. gst-launch-1.0 v4l2src ! video/x-h264,width=640,height=480 ! fakesink This sits and crunches for a long time before it fails with error: Internal data flow error. error: streaming task paused, reason not-negotiated (-4) Interestingly, using "video/x-h264,width=640,height=640" (width==height) it crunches for a long time (~20s) and then starts emitting frames. It has been surmised at [2] that this is due to a faulty probing loop in gstv4l2object.c (cf. [3]). Since looking into the issue is hard without access to the hardware I found that the synthetic vivi.ko driver exhibits problems with stepwise as well. I found it impossible to pick a resolution other than width=1920,height=500 using vivi. There’s seems to be an issue with the probing logic. Is it really necessary to iterate through 14 million modes to pick a framesize+fps combination? [1] https://github.com/raspberrypi/linux/commit/983dcb1d#diff-221e79295f1bf120ffa5fdfae25d2e38R1130 [2] http://www.raspberrypi.org/phpBB3/viewtopic.php?p=521235#p521235 [3] http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gstv4l2object.c#n1866
Bug #724521 seems to be related. Why hadn’t I found that before?
Note, the frame size code is not new at all. We'll have a look later. Also many drivers seems to offer steps where they should offer range, if this is the case, we'll got for the driver fix, not adding more hacks in GStreamer. Again, tha GStreamer code is old and probably need review.
The driver developers have elected to hide much of the sensor modes and a lot of scaling and cropping and binning is hidden from the user and implicit in output size and fps. So where the sensor there can deliver 2592x1944@15, 1296x972@41, 1296x730@49, 640x480@60, 640x480@90, they just do any scaling and cropping implicitly and on the hardware, and then allow the user to choose anything between 16,16 and 2592,1944 in steps of two, where modes are picked behind the scenes on a best-fit best-effort basis. There’s a description also at [1] and [2]. I can’t tell if that is the most optimal way to offer frames formats. [1] http://www.raspberrypi.org/phpBB3/viewtopic.php?p=520078#p520078 [2] http://www.raspberrypi.org/phpBB3/viewtopic.php?p=520132#p520132
Just looked again, and this seems fair. I think we need to have some threshold for step size, or when large amount of discrete modes may be generated. What I found (at least for sources) is that good practice is to roundup to the step. This way, if GStreamer needs 16x17, the driver would give 16x18, and GStreamer could use an implicit crop using stride of 18 and with of 17. For larger steps, if crop meta is supported, we could center the crop. A lot of HW seems to have a 16bit steps, though loosing up to 8 pixels border, if centers, is not that of a problem. Most capture HW already crops of lot due to lenses artifact. So I agree, we need to look into something smarter here, and avoid probing all of the possibilities. I haven't looked deeply in that code yet, but I have RPi and should be able to dig into that at some point. Obviously if someone has more time then me, please let me know.
Thanks for looking into this. Just a note, it sounded like they want to add cropping feature to this driver later. And a question, what I didn't understand, why is it necessary to probe all possible mode combinations ahead of time anyway? If I ask for 640x480 at 60fps, why does it need to iterate through all of them to build these massive data structures? Why can't it just jump straight to that requested mode?
(In reply to comment #5) > And a question, what I didn't understand, why is it necessary to probe all > possible mode combinations ahead of time anyway? That is also a good question that I need to look at. A lot of the GStreamer V4L2 code has been implemented on top of broken drivers, so it may explain some strange behavior. Though, these days, I want to encourage fixing drivers, and have GStreamer implement as few quirks as possible. > > If I ask for 640x480 at 60fps, why does it need to iterate through all of them > to build these massive data structures? Why can't it just jump straight to that > requested mode? The iteration probably occurs before the element is aware of the downstream caps. In general, an element need to probe the HW to offer a valid set of caps in order to do the negotiation.
Can we mark this as a duplicate of bug #724521 or are there any aspects in this bug which are not covered by the other bug already?
I think it's a dupe. If you have all info close this one.
Great, thanks. *** This bug has been marked as a duplicate of bug 724521 ***