After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 512345 - gstalpha: chroma-key should not set alpha to 0 if brightness is near maximum or minimum
gstalpha: chroma-key should not set alpha to 0 if brightness is near maximum ...
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
git master
Other Linux
: Normal normal
: 0.10.9
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2008-01-27 10:20 UTC by Sebastian Keller
Modified: 2008-05-29 11:30 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
the patch I used in the bug description (415 bytes, patch)
2008-01-27 10:23 UTC, Sebastian Keller
committed Details | Review
some pictures showing the problem (474.58 KB, image/png)
2008-01-28 22:23 UTC, Sebastian Keller
  Details

Description Sebastian Keller 2008-01-27 10:20:33 UTC
I tried using the "alpha" filter in combination with videomixer to get something like a bluescreen effect working for my old webcam (Replace one color of a stream with another stream). Since the old webcam tends to have problems with lightsources or bright reflections it often has a bright white in the picture. As a result not just blue got replaced but also many of the white light reflections in the picture.
Then i changed gst_alpha_chroma_key_i420 in gstalpha.c to always use an alpha value of 255 if the brightness(y) is near the maximum or near the minimum (which was also a problem; black got replaced, too). I did tried to check if it equals 229 (which seems to be the maximum) or 0 first, but saw that it did not catch everything, so i checked if it is >= 225 or <= 5 which worked even better.
After that the result was as expected and no more bright lights or dark shadows were replaced, only blue was replaced.
If i understand it right, thats because one can't determine the color if the brightness value is at maximum or minimum and other colors are more likely to get matched if the brightness is near maximum or minimum which leads to lots of false matches.
Maybe somebody who completely understands the algorithm could create a better patch since my version is mostly trial&error ;)
Comment 1 Sebastian Keller 2008-01-27 10:23:27 UTC
Created attachment 103810 [details] [review]
the patch I used in the bug description
Comment 2 Jan Schmidt 2008-01-27 23:31:18 UTC
The argument sounds plausible. Wim coded the original algorithm, perhaps he has a better idea - even though it was 2004.

The single if statement seem strange - wouldn't it work better if the decision was made for each y value individually? Also, is it worth making it configurable in a property? Thirdly, whatever fix we end up with should probably happen in the AYUV-format chromakey handler too.
Comment 3 Wim Taymans 2008-01-28 17:17:00 UTC
I can't seem to make it fail using videotestsrc (which presumably also has black and white areas). Can you provide a sample of a picture of the camera?
Comment 4 Sebastian Keller 2008-01-28 22:23:05 UTC
Created attachment 103916 [details]
some pictures showing the problem

First I modified my patch a bit for these pictures, since it was causing some problems with videotestsrc, it just checks for (y11 >= 225 || y12 >= 225 || y21 >= 225 || y22 >= 225) now.

The command i used for testing:
gst-launch-0.10 \
videotestsrc ! "video/x-raw-yuv, width=360, height=296, framerate=(fraction)15/2" ! ffmpegcolorspace ! \
alpha alpha=0.0 method=blue angle=50 noise_level=0 ! videomixer name=mix ! ffmpegcolorspace ! ximagesink \
filesrc location="/usr/share/pixmaps/backgrounds/gnome/nature/FreshFlower.jpg" ! decodebin2 ! \
ffmpegcolorspace ! videoscale ! "video/x-raw-yuv, width=360, height=296" ! ffmpegcolorspace ! mix.

(simply replace videotestsrc with v4lsrc for webcam)
Comment 5 Sebastian Dröge (slomo) 2008-05-29 08:27:11 UTC
Wim, could you have a look at the patch again? It seems to be still needed...
Comment 6 Wim Taymans 2008-05-29 11:30:43 UTC
Improved for both AYUV and I420 with configurable limits for dark and brightness.


        Based on patch by: Sebastian Keller <sebastian-keller at gmx dot de>

        * gst/alpha/gstalpha.c: (gst_alpha_class_init), (gst_alpha_init),
        (gst_alpha_set_property), (gst_alpha_get_property),
        (gst_alpha_chroma_key_ayuv), (gst_alpha_chromakey_row_i420):
        Try to skip pixels or areas that are too dark or too bright for us to do
        meaningfull color detection.
        Added properties to control the sensitivity to light and darkness.
        Added some small cleanups. Fixes #512345.