After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 768195 - v4l2: Add support for device capability
v4l2: Add support for device capability
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
git master
Other Linux
: Normal normal
: 1.9.1
Assigned To: Reynaldo H. Verdejo Pinochet
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2016-06-29 15:40 UTC by Nicolas Dufresne (ndufresne)
Modified: 2016-07-11 01:38 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
v4l2: use opened device caps instead of physical device ones (12.54 KB, patch)
2016-07-02 09:21 UTC, Reynaldo H. Verdejo Pinochet
committed Details | Review
use caps intersect API instead of re-implementing GST_CAPS_INTERSECT_FIRST (1.97 KB, patch)
2016-07-02 09:23 UTC, Reynaldo H. Verdejo Pinochet
committed Details | Review
Fix capture of multiplanar devices (1.86 KB, patch)
2016-07-08 17:38 UTC, Luis de Bethencourt
committed Details | Review
v4l2: Also copy device_caps in gst_v4l2_dup (911 bytes, patch)
2016-07-11 01:36 UTC, Nicolas Dufresne (ndufresne)
committed Details | Review

Description Nicolas Dufresne (ndufresne) 2016-06-29 15:40:59 UTC
At some point in time (v4l2 in gst is very old) the capability structure in V4L2 was extended in a way it broke the ABI. A driver now uses the v4l2_capability::capabilities as the intersection of all capabilities by adding V4L2_CAP_DEVICE_CAPS flags there. In GStreamer, when this flag is present, we always want to look at the v4l2_capability::device_caps. A suggestion while porting would be to check if this flags is present, if not, copy the capabilities into device_caps. And then, everywhere in the code, replace capabilities with device_caps.
Comment 1 Luis de Bethencourt 2016-06-29 15:48:27 UTC
This issue can be seen when using gst v4l2{src,sink} elements and vivid. vivid is the Virtual Video Test Driver.

$ gst-launch-1.0 v4l2src device=/dev/video1 ! v4l2sink device=/dev/video2

Fails to negotiate the caps, it should work when passing input_types=0x9 as module parameter to vivid. Looking at the logs with GST_DEBUG=*CAPS*:5 the only difference is in the colorimetry. colorimetry=(string)2:4:7:1 for v4l2src and colorimetry=(string)bt601 for v4l2sink.
Comment 2 Nicolas Dufresne (ndufresne) 2016-06-29 16:20:03 UTC
Not sure it's a single bug here, the colorimetry issue would mean both src and sink aren't colorimetry compatible and you need a color converter. What type 0x9 ?
Comment 3 Reynaldo H. Verdejo Pinochet 2016-06-30 05:04:03 UTC
AFAIU input_types=0x9 asks vivid to create a TV tuner and an S-Video input for the first driver instance, 4 inputs are created by default according to the docs. To reproduce Luis's example I'm actually using: n_devs=2 node_types=0x1,0x100 num_inputs=1 input_types=0x2 num_outputs=1 output_types=0x0 for two instances, one S-Video input and one S-Video output. More info here:

https://www.kernel.org/doc/Documentation/video4linux/vivid.txt
Comment 4 Reynaldo H. Verdejo Pinochet 2016-07-02 09:21:24 UTC
Created attachment 330767 [details] [review]
v4l2: use opened device caps instead of physical device ones

As expected this doesn't solve our colorimetry issue as the caps
are at best case trimmed by the fact we are using the opened device
instead of the physical device ones but should go a long way towars
doing the right thing. Pretty rough patch only mildly tested. Please
review.
Comment 5 Reynaldo H. Verdejo Pinochet 2016-07-02 09:23:07 UTC
Created attachment 330768 [details] [review]
use caps intersect API instead of re-implementing GST_CAPS_INTERSECT_FIRST

Bonus patch. Spotted this one while working around the caps handling code.
Comment 6 Nicolas Dufresne (ndufresne) 2016-07-07 21:46:50 UTC
Attachment 330767 [details] pushed as 4b16b16 - v4l2: use opened device caps instead of physical device ones
Comment 7 Nicolas Dufresne (ndufresne) 2016-07-07 21:50:18 UTC
And git bz failed on the second:

commit dfdc15c548a0df957bfd9cb6944f4f099bd52b03
Author: Reynaldo H. Verdejo Pinochet <reynaldo@osg.samsung.com>
Date:   Fri Jul 1 22:53:33 2016 -0700

    v4l2src: use gst_caps_intersect_full in negotiate()

And 1.8

commit d6c8cb63859368e7100b0002354ad742e4286eba
Author: Reynaldo H. Verdejo Pinochet <reynaldo@osg.samsung.com>
Date:   Sat Jul 2 01:56:07 2016 -0700

    v4l2: use opened device caps instead of physical device ones

The other one does not fix a bug, it's more like a cleanup, that's why I don't backport here.
Comment 8 Nicolas Dufresne (ndufresne) 2016-07-07 21:50:43 UTC
Thanks.
Comment 9 Luis de Bethencourt 2016-07-08 17:38:45 UTC
Created attachment 331105 [details] [review]
Fix capture of multiplanar devices

This can be tested by doing:
$ sudo modprobe vivid node_types=0x101 output_types=0x1 input_types=0x3 num_inputs=1 num_outputs=1 multiplanar=2
$ gst-launch-1.0 v4l2src device=/dev/video2 io-mode=dmabuf ! fakesink

Before the patch:
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to enumerate possible video formats device '/dev/video2' can work with

After the patch:
Playback \o/


This happened because in sys/v4l2/gstv4l2object.c:gst_v4l2_object_fill_format_list():
if (v4l2_ioctl (v4l2object->video_fd, VIDIOC_ENUM_FMT, format) < 0) {

Would send the wrong format, and the vivid driver would return ENOTTY since you can't do VIDIO_ENUM_FMT with single-planer on a multiplanar device, or viceversa.
Comment 10 Nicolas Dufresne (ndufresne) 2016-07-08 17:47:15 UTC
Review of attachment 331105 [details] [review]:

Haha, funny, we relied on this bug. Look good.
Comment 11 Luis de Bethencourt 2016-07-08 18:11:13 UTC
Review of attachment 331105 [details] [review]:

commit eaa01071e0f5eea915edfd3510f6f3b71930576c
Author: Luis de Bethencourt <luisbg@osg.samsung.com>
Date:   Fri Jul 8 17:28:19 2016 +0000

    v4l2: fix multiplanar capture

    After switching to using V4L2_CAP_DEVICE_CAPS we lost support for
    multiplanar device types. After some research, it looks like
    vcap.capabilities treated the multiplanar flag of output and capture
    devices equally, but not the new device_caps.

    https://bugzilla.gnome.org/show_bug.cgi?id=768195
Comment 12 Nicolas Dufresne (ndufresne) 2016-07-11 01:30:26 UTC
Apparently you also forgot to copy the new member device_caps in the function gst_v4l2_dup() used to create shared v4l2object. In M2M, we don't probe a second time the device, it would be a waste, we simply copy the information and adapt what need to be adapted. The broke every M2M elements.
Comment 13 Nicolas Dufresne (ndufresne) 2016-07-11 01:36:48 UTC
Created attachment 331174 [details] [review]
v4l2: Also copy device_caps in gst_v4l2_dup

This fixes regression where M2M error out saying they have no output
format (the V4L2 CAPTURE side).
Comment 14 Nicolas Dufresne (ndufresne) 2016-07-11 01:37:59 UTC
Attachment 331174 [details] pushed as 5a9c0aa - v4l2: Also copy device_caps in gst_v4l2_dup