GNOME Bugzilla – Bug 512345
gstalpha: chroma-key should not set alpha to 0 if brightness is near maximum or minimum
Last modified: 2008-05-29 11:30:43 UTC
I tried using the "alpha" filter in combination with videomixer to get something like a bluescreen effect working for my old webcam (Replace one color of a stream with another stream). Since the old webcam tends to have problems with lightsources or bright reflections it often has a bright white in the picture. As a result not just blue got replaced but also many of the white light reflections in the picture. Then i changed gst_alpha_chroma_key_i420 in gstalpha.c to always use an alpha value of 255 if the brightness(y) is near the maximum or near the minimum (which was also a problem; black got replaced, too). I did tried to check if it equals 229 (which seems to be the maximum) or 0 first, but saw that it did not catch everything, so i checked if it is >= 225 or <= 5 which worked even better. After that the result was as expected and no more bright lights or dark shadows were replaced, only blue was replaced. If i understand it right, thats because one can't determine the color if the brightness value is at maximum or minimum and other colors are more likely to get matched if the brightness is near maximum or minimum which leads to lots of false matches. Maybe somebody who completely understands the algorithm could create a better patch since my version is mostly trial&error ;)
Created attachment 103810 [details] [review] the patch I used in the bug description
The argument sounds plausible. Wim coded the original algorithm, perhaps he has a better idea - even though it was 2004. The single if statement seem strange - wouldn't it work better if the decision was made for each y value individually? Also, is it worth making it configurable in a property? Thirdly, whatever fix we end up with should probably happen in the AYUV-format chromakey handler too.
I can't seem to make it fail using videotestsrc (which presumably also has black and white areas). Can you provide a sample of a picture of the camera?
Created attachment 103916 [details] some pictures showing the problem First I modified my patch a bit for these pictures, since it was causing some problems with videotestsrc, it just checks for (y11 >= 225 || y12 >= 225 || y21 >= 225 || y22 >= 225) now. The command i used for testing: gst-launch-0.10 \ videotestsrc ! "video/x-raw-yuv, width=360, height=296, framerate=(fraction)15/2" ! ffmpegcolorspace ! \ alpha alpha=0.0 method=blue angle=50 noise_level=0 ! videomixer name=mix ! ffmpegcolorspace ! ximagesink \ filesrc location="/usr/share/pixmaps/backgrounds/gnome/nature/FreshFlower.jpg" ! decodebin2 ! \ ffmpegcolorspace ! videoscale ! "video/x-raw-yuv, width=360, height=296" ! ffmpegcolorspace ! mix. (simply replace videotestsrc with v4lsrc for webcam)
Wim, could you have a look at the patch again? It seems to be still needed...
Improved for both AYUV and I420 with configurable limits for dark and brightness. Based on patch by: Sebastian Keller <sebastian-keller at gmx dot de> * gst/alpha/gstalpha.c: (gst_alpha_class_init), (gst_alpha_init), (gst_alpha_set_property), (gst_alpha_get_property), (gst_alpha_chroma_key_ayuv), (gst_alpha_chromakey_row_i420): Try to skip pixels or areas that are too dark or too bright for us to do meaningfull color detection. Added properties to control the sensitivity to light and darkness. Added some small cleanups. Fixes #512345.