GNOME Bugzilla – Bug 166207
[PATCH] [ffmpegcolorspace] AYUV->RGB is broken
Last modified: 2005-02-25 08:41:36 UTC
Test case: gst-launch videotestsrc ! alpha ! ffmpegcolorspace ! ximagesink Result: Observe that the picture is half the expected width (the rest being black), and the colors are rather weird. Expected result: The normal videotestsrc result. Workaround: Converting to another YUV format, and then to RGB seems to fix the problem. Eg, this works: gst-launch videotestsrc ! alpha ! ffmpegcolorspace ! video/x-raw-yuv,format=(fourcc)AYUV ! ffmpegcolorspace ! ximagesink
Whops, sorry. AYUV in the second gst-launch pipeline should have been I420, or nearly anything but AYUV.
I've done some digging, and found that in the AYUV->RGB555 case img_convert does a temporary AYUV->RGBA32 conversion, and converts that to RGB555. While doing this, it looks up the AYUV->RGBA32 from convert_table, and uses the convert function specified therein. This function is ayuv4444_to_rgba32. Now, for some weird reason, BPP and RGBA_OUT are defined as for RGB555 output instead of RGBA32 (most probably due to the messy way imgconvert_template.h is included around). This breaks the conversion badly. Further investigation showed, that ayuv4444_to_rgba32 gets defined as soon as the !defined(FMT_RGBA32) && defined(RGBA_OUT) conditions apply. Ie, at the time of the first rgb definition in imgconvert.c. The fix here is to only define ayuv4444_to_rgba32 when FMT_RGBA32 is defined. This means splitting the large #if !defined(FMT_RGBA32) && defined(RGBA_OUT) before ayuv4444_to_rgba32 into two. End it just before ayuv4444_to_rgba32, and then do another condition: #if defined(FMT_RGBA32). I'll attach a patch that does this, after some testing.
Created attachment 37529 [details] [review] Patch fixing the problem This does exactly what I outlined in my previous comment. AYUV can get converted to RGBA32 and RGB24 only, all other RGB formats are converted from one of these two. I tested both AYUV->RGBA32 and ->RGB24 and they look correct, as does AYUV->RGBA32->RGB555.
Nice catch! Applied, thanks.