GNOME Bugzilla – Bug 774576
opencv: caps to IplImage format conversion fails for sparse formats
Last modified: 2016-11-30 02:23:03 UTC
gstopencvutils has utility to extract the number of channel and depth from caps. This is later used to select an appropriate IplImage format. This utility does not handle sparse formats well. It will report 3 channels when it should report 4 channels.
Created attachment 340032 [details] [review] opencv: account for sparse formats when converting caps to IPlImage format This demonstrates a use case of https://bugzilla.gnome.org/show_bug.cgi?id=773433
Here is some background about how images are handled in opencv. The various opencv image containers or headers carry the following information: - number of channels (usually 1 to 4) - depth (8, 16, 32, 64...); all channels have the same depth. The channel layout (BGR vs RGB) is not carried... This gives us the following list of supported image formats: CV_8UC1, CV_8UC2, CV_8UC3, CV_8UC4 CV_8SC1, CV_8SC2, CV_8SC3, CV_8SC4 CV_16UC1, CV_16UC2, CV_16UC3, CV_16UC4 CV_16SC1, CV_16SC2, CV_16SC3, CV_16SC4 CV_32SC1, CV_32SC2, CV_32SC3, CV_32SC4 CV_32FC1, CV_32FC2, CV_32FC3, CV_32FC4 CV_64FC1, CV_64FC2, CV_64FC3, CV_64FC4 Where the first part of the format name is the depth followed by a digit representing the number of channels. Note that opencv supports more that 4 channels. The opencv algorithms don't all support all the image types. For example findChessboardCorners() supports only 8 bits formats (gray scale and color). And, typically, this algorithm will convert the image to gray scale before proceeding. It will do so with something like this: cvtColor(srcImg, destImg, CV_BGR2GRAY); The conversion will work on any BGR format (BGR, BGRA, BGRx). The extra channel(s) will be ignored. It will also produce a result for any RGB format. The result will be "wrong" to the human eye and might affect some algorithms (not findChessboardCorners() afaik...). This is due to how RGB gets converted to gray where each color has a different weight. Another example is the 2D rendering API. It work with RGB but the colors will be wrong. The bad thing is that it is not possible to change the "default" BGR format. Safest is to not assume that RGB will work and always convert to BGR. That said, the current opencv gstreamer elements all accept BGR and RGB caps ! Some have restrictions but if a format is supported then both BGR and RGB layouts will be supported. The patch that I am about to push does not address the above. Instead it makes the situation worse ;) by adding support for the x layouts (BGRx, RGBx). Testing shows that x layouts work with expected color errors when using RGB layouts.
Created attachment 340796 [details] [review] opencv: account for sparse/padded formats when converting caps to cv image type
Created attachment 340804 [details] [review] opencv: account for sparse/padded formats when converting caps to cv image type
Created attachment 340805 [details] [review] opencv: add opencv image format documentation
I will review this on Monday. If I forget, remind me via email.
Review of attachment 340804 [details] [review]: Looks good, I have conflicting patch here, will try to merge it.
Review of attachment 340805 [details] [review]: Looks good.
Attachment 340805 [details] pushed as 4664fc0 - opencv: add opencv image format documentation