GNOME Bugzilla – Bug 792435
v4l2src selecting suboptimal colorspace
Last modified: 2018-01-12 14:22:22 UTC
Created attachment 366676 [details] debug output I was expecting the following pipeline to negociate NV12 directly from the device (whose driver supports NV12), but for an unknown reason v4l2 selects YVYU: $ GST_DEBUG=v4l2*:5 gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! queue name=qinput1 ! videoconvert ! videoscale ! "video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30, colorimetry=(string)bt709, pixel-aspect-ratio=1/1" ! fakesink This will imply an unnecessary colorspace conversion by videoconvert. Manually requesting NV12 does work though: $ GST_DEBUG=v4l2*:5 gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! queue name=qinput1 ! "video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30, colorimetry=(string)bt709, pixel-aspect-ratio=1/1" ! fakesink -v Why is v4l2src not choosing NV12 in the first place ?
Looking into this, I don't think it's a regression of the new negotiation code. But please, give it a try on 1.12 or less to confirm. Currently, the format is totally ignored when picking up the "best" option. Though, the list we iterate over has been sorted with a made up rank for each color format, where YUY2 gets 1010 as a rank, while NV12 gets 50. That may explain why YUY2 is picked. Just to recap the current state: - GstV4L2Object create a sorted formats list, with YUY2 first - GstV4l2Object create 1 caps structure per format (in the sorted order) - GstV4l2Src does gst_caps_intersect_full (peercaps, thiscaps, _FIRST); - GstV4l2Src iterate over the structure (and fixate) to find a format What I observe is that the result of the intersection maintains the GstV4l2Object chosen order, loosing the "preferred" format that was prepended by videoconvert. So the chosen format shall be optional in regard to the size (with videoscale) but nothing is done for videoconvert. Maybe we need to add a format preference to the preferred structure, and sort the fixated structure with matching format first.
My analyses was not correct, it's only when sorting that order changes from what downstream preferred. In fact, between "fixating caps" and "sorted and normalized caps" I clearly see that the order was reverted. That looks like a error, and likely a regression in fact.
I think I found the problem, just wrote this little test to find out that g_list_insert_sorter() will prepend when the compare function return equals (0). That means we reverse the order. It's funny, because I ready have that handled for the case both structure are smaller then preferred size (I just append by returning 1).
Created attachment 366689 [details] [review] v4l2src: Maintain downstream caps order The g_list_insert_sorted() will behave like prepend when the compare function returns 0. In our case, we want to maintain the order hence append. This fixes this issue and improve the sorting algorithm to make a 10x10 prefered over 10x200 with a preference of 10x8 (and similar cases which was badly handled). This fixes generally fixes issue were a sub-optimal format / size is picked.
Great, seems to work as expected ! $ gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! queue name=qinput1 ! videoconvert ! videoscale ! "video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30, colorimetry=(string)bt709, pixel-aspect-ratio=1/1" ! fakesink -v Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)bt709, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstQueue:qinput1.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)bt709, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstQueue:qinput1.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)bt709, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)bt709, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)bt709, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)bt709, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)bt709, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)bt709, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)bt709, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)bt709, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive Got EOS from element "pipeline0". Execution ended after 0:00:00.004407904 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ...
Thanks for testing. Attachment 366689 [details] pushed as 21b01ac - v4l2src: Maintain downstream caps order