After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 545033 - [v4l2src] Add support for camera specific formats
[v4l2src] Add support for camera specific formats
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
0.10.8
Other Linux
: Normal normal
: 0.10.14
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2008-07-27 20:40 UTC by Hans de Goede
Modified: 2009-01-23 09:39 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
PATCH adding support for using libv4l to the v4l2 plugin (18.25 KB, patch)
2008-07-27 20:41 UTC, Hans de Goede
needs-work Details | Review
With cleanups and configure checks (18.97 KB, patch)
2008-07-30 10:24 UTC, Bastien Nocera
none Details | Review
Updated patch (23.10 KB, patch)
2008-10-25 00:18 UTC, Bastien Nocera
committed Details | Review

Description Hans de Goede 2008-07-27 20:40:53 UTC
Short intro: I'm a long time Linux developer currently working on improving webcam support in Linux, see:
http://fedoraproject.org/wiki/Features/BetterWebcamSupport

I'm one of the authors of the v4l2 rewrite of the gspca usb webcam driver framework (which supports more then 100 different cams), this v4l2 rewrite has been merged into the 2.6.27 kernel and thus will become available in the official Linux kernel soon.

One of the parts of the v4l2 rewrite has been removing conversion from various manufacturer cam specific video formats to more normal videoformats from the drivers, as this really does not belong in userspace.

As a result of this the gspca subdrivers can generate raw video frames in the following formats:

#define V4L2_PIX_FMT_SN9C10X  v4l2_fourcc('S', '9', '1', '0') /* SN9C10x compressed bayer
#define V4L2_PIX_FMT_SPCA501  v4l2_fourcc('S', '5', '0', '1') /* YUYV per line 
#define V4L2_PIX_FMT_SPCA505  v4l2_fourcc('S', '5', '0', '5') /* YYUV per line 
#define V4L2_PIX_FMT_SPCA508  v4l2_fourcc('S', '5', '0', '8') /* YUVY per line 
#define V4L2_PIX_FMT_SPCA561  v4l2_fourcc('S', '5', '6', '1') /* compressed GBRG bayer
#define V4L2_PIX_FMT_PAC207   v4l2_fourcc('P', '2', '0', '7') /* compressed BGGR bayer

As we do not want to see each application implement (nor having to implement) support for these various formats, as we like to have one code base and only one code base for these formats. A library has been written (with me as the main author) to convert from these formats to BGR24 or YUV420. For more on this library see:
http://hansdegoede.livejournal.com/3636.html

The attached patch adds support to the gstreamer v4l2 plugin to use this library if available on the system.

Note that currently there is an ifdef to handle compilation with/without the library but this is hard defined in the header file. I've done it this way as my autoconf skills are almost non existent so I thought it would be good to let someone else write the autoconf detection of libv4l.

You can get the latest version of libv4l here:
http://people.atrpms.net/~hdegoede/libv4l-0.3.7.tar.gz

Note that this library is an official part of the v4l-dvb project, we just haven't gotten around yet to making a homepage for it at linuxtv.org, but if you look for example at the official v4l-dvb mercurial tree:
http://linuxtv.org/hg/v4l-dvb
Under v4l2-apps/lib/libv4l you will find libv4l there, a proper homepage is on my todo.

A note about the patch, the patch is actually quite simple, as libv4l has been designed for quick porting of existing apps to libv4l, for this reason libv4l fully mimicks the raw /dev/video interface (and if no conversion is needed and for non relevant calls like setting ctrl's, passes through all calls unmodified).

All that the patch does and all that needs to be changed to use libv4l is replace the relevant xxx calls on /dev/video with v4l2_xxx calls.

The only special thing about the patch is that it passes the V4L2_ENABLE_ENUM_FMT_EMULATION flag to libv4l2 on open, this patch tells libv4l2 to report the "emulated" formats in the ENUM_FMT ioctl results. Which libv4l2 does not do be default so that ENUM_FMT only reports the really supported formats (a TRY_FMT with one of the 2 supported emulated formats can then be done to see if conversion is supported for the specific device). Given the working of the v4l2 plugin, enabling the ENUM_FMT emulation is needed.

I've successfully tested this patch with cheese and 2 spca561 cams (quite popular IC, used by both logitech and creative), 2 pac207 cams (another quite popular IC), 3 SN9C10X cams (also very popular) and 1 spca501 cam. All of which do not work without this patch, as gstreamer does not support their proprietary video formats (and rightfully so).
Comment 1 Hans de Goede 2008-07-27 20:41:58 UTC
Created attachment 115388 [details] [review]
PATCH adding support for using libv4l to the v4l2 plugin
Comment 2 Bastien Nocera 2008-07-28 18:54:19 UTC
Note that libv4l is LGPL, so the license matches what we expect.

Note that your patch is missing configure.ac changes to detect libv4l2. You should add a pkg-config file to your upstream library, and think whether libv4l should be made a hard or optional compile-time dependency. I personally don't see many reasons why we shouldn't be making it a hard dependency.

Also:
> +++ gst-plugins-good-0.10.8/sys/v4l2/Makefile.in	2008-07-27 21:50:12.000000000 +0200

Don't diff generated files
Comment 3 Jan Schmidt 2008-07-28 19:36:20 UTC
v4l2src currently builds on newer OpenSolaris releases, it'd be nice if that continues working. How portable is libv4l?
Comment 4 Hans de Goede 2008-07-29 21:58:45 UTC
(In reply to comment #2)
> Note that your patch is missing configure.ac changes to detect libv4l2. You
> should add a pkg-config file to your upstream library, and think whether libv4l
> should be made a hard or optional compile-time dependency. I personally don't
> see many reasons why we shouldn't be making it a hard dependency.
> 

I've just done a new release which includes .pc files:
http://people.atrpms.net/~hdegoede/libv4l-0.3.8.tar.gz

This is also currently building for Fedora development (rawhide).

I was hoping someone with better autoconf skills then me could do the autoconf part, if not let me know and I'll do my best.
Comment 5 Hans de Goede 2008-07-29 22:08:14 UTC
(In reply to comment #3)
> v4l2src currently builds on newer OpenSolaris releases, it'd be nice if that
> continues working. How portable is libv4l?
> 

The base code should be pretty portable but libv4l was not designed / written with portability in mind, on the contrary in several occasions I've done things which I knew to be Linux specific, and I thought those were ok as well v4l sounds rather Linux specific to begin with.

So there are several issues I can think of from my head:
1) It needs linux/videodev.h and linux/videodev2.h, but I guess that if
   opensolaris has some kind of v4l2 interface it may have these.

2) It is designed for all its v4l2_xxx functions to behave just like
   their regular xxx counter parts (open, close, ioctl, mmap, etc), this
   is because libv4l can also be used without modifying apps by loading it
   through a libc-wrapper which can be loaded with LD_PRELOAD and will then
   call v4l2_open for all open calls, etc.

   In order to be able todo for example a read() call where it is itself
   being called as read() it uses syscall(SYS_read, ...), I don't know if
   this will work on opensolaris, and for mmap it uses SYS_mmap2 !

So all in all this might be a problem, but if you look at the patch you will see the patch already contains a single #ifdef to switch between compilation with and without libv4l so compiling without libv4l on opensolaris is easy. And as long as opensolaris doesn't have any webcam drivers generating webcam custom / proprietary formats there is little use / sense in using libv4l on opensolaris.
Comment 6 Jan Schmidt 2008-07-30 06:53:14 UTC
(In reply to comment #5)
> (In reply to comment #3)
> > v4l2src currently builds on newer OpenSolaris releases, it'd be nice if that
> > continues working. How portable is libv4l?
> > 
> 
> The base code should be pretty portable but libv4l was not designed / written
> with portability in mind, on the contrary in several occasions I've done things
> which I knew to be Linux specific, and I thought those were ok as well v4l
> sounds rather Linux specific to begin with.
> 
> So there are several issues I can think of from my head:
> 1) It needs linux/videodev.h and linux/videodev2.h, but I guess that if
>    opensolaris has some kind of v4l2 interface it may have these.
> 

It has videodev2.h, but I guess they didn't want to introduce a /usr/include/linux subdir - so videodev2.h is found in /usr/include/sys.

At the moment, Solaris only supports usbvc camera devices, and I've only seen UYVY and JPEG output, neither of which are 'camera specific'

I think we definitely want to keep the #ifdef code paths and not add a hard dependency on libv4l.
Comment 7 Bastien Nocera 2008-07-30 10:24:38 UTC
Created attachment 115551 [details] [review]
With cleanups and configure checks
Comment 8 Hans de Goede 2008-07-30 17:46:09 UTC
(In reply to comment #7)
> Created an attachment (id=115551) [edit]
> With cleanups and configure checks
> 

Thanks!

Looks good, but I wonder about:

+if test x$HAVE_GST_V4L2 = xyes; then
+  PKG_CHECK_MODULES(LIBV4L2, libv4l2, [have_libv4l2=yes], [have_libv4l2=no])
+  AC_DEFINE([HAVE_LIBV4L2], test x$have_libv4l2 = xyes,
+	    [Whether libv4l2 is available for video buffer conversion])
+fi
+

You don't define HAVE_GST_V4L2 anywhere, shouldn't you add some code to configure.ac to get a cmdline option defining this?
Comment 9 Bastien Nocera 2008-07-30 17:51:20 UTC
It detects if we have a libv4l2.pc, if not, doesn't define HAVE_LIBV4L2. I don't see the problem here...
Comment 10 Hans de Goede 2008-07-30 17:55:44 UTC
As said I'm not good in autoconf, but as I read it it first checks that $HAVE_GST_V4L2, erm I see it now, that is if the v4l2 plugin is build at all, never mind and thanks for the autoconf patch!
Comment 11 Sebastian Dröge (slomo) 2008-10-20 11:09:01 UTC
What's the state on this now? It seems that Fedora and Ubuntu are using this now (https://bugs.launchpad.net/ubuntu/+source/libv4l/+bug/260918 https://bugzilla.redhat.com/show_bug.cgi?id=456825 ) and we should probably get it in because from what I understood some webcams need it to work with kernels newer than 2.6.27 or something...
Comment 12 Bastien Nocera 2008-10-25 00:18:29 UTC
Created attachment 121321 [details] [review]
Updated patch

This one applies cleanly to the 0.10.11 release (just need to snip the generated bits).
Comment 13 Sebastian Dröge (slomo) 2008-11-04 12:30:03 UTC
2008-11-04  Sebastian Dröge  <sebastian.droege@collabora.co.uk>

        Patch by: Bastien Nocera <hadess at hadess dot net>,
                  Hans de Goede <jwrdegoede at fedoraproject dot org>

        * configure.ac:
        * sys/v4l2/Makefile.am:
        * sys/v4l2/gstv4l2src.c: (gst_v4l2src_get_read):
        * sys/v4l2/v4l2_calls.c: (gst_v4l2_get_capabilities),
        (gst_v4l2_fill_lists), (gst_v4l2_open), (gst_v4l2_close),
        (gst_v4l2_get_norm), (gst_v4l2_set_norm), (gst_v4l2_get_frequency),
        (gst_v4l2_set_frequency), (gst_v4l2_signal_strength),
        (gst_v4l2_get_attribute), (gst_v4l2_set_attribute),
        (gst_v4l2_get_input), (gst_v4l2_set_input):
        * sys/v4l2/v4l2_calls.h:
        * sys/v4l2/v4l2src_calls.c: (gst_v4l2_buffer_finalize),
        (gst_v4l2_buffer_new), (gst_v4l2_buffer_pool_finalize),
        (gst_v4l2_buffer_pool_new), (gst_v4l2_buffer_pool_activate),
        (gst_v4l2src_fill_format_list),
        (gst_v4l2src_probe_caps_for_format_and_size),
        (gst_v4l2src_probe_caps_for_format), (gst_v4l2src_grab_frame),
        (gst_v4l2src_set_capture), (gst_v4l2src_capture_init),
        (gst_v4l2src_capture_start), (gst_v4l2src_capture_stop),
        (gst_v4l2src_get_nearest_size):
        Optionally use libv4l to access v4l2 devices. Fixes bug #545033.
Comment 14 Götz Waschk 2009-01-23 09:35:54 UTC
This patch is still needed for gst-plugins-good 0.10.13.
Comment 15 Sebastian Dröge (slomo) 2009-01-23 09:39:53 UTC
(In reply to comment #14)
> This patch is still needed for gst-plugins-good 0.10.13.

Yes, because 0.10.12 and 0.10.13 are releases based on 0.10.11 with just a single patch for a security issue.

0.10.14 will have a fix for this bug included.