After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 722656 - videoconvert: scoring system ranks colour loss very low
videoconvert: scoring system ranks colour loss very low
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-base
git master
Other Linux
: Normal normal
: 1.2.3
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2014-01-20 21:17 UTC by Michael Sheldon
Modified: 2014-01-24 11:27 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Patch to adjust the weighting of videoconvert's scoring mechanism to preserve colour information (1.00 KB, patch)
2014-01-20 21:17 UTC, Michael Sheldon
needs-work Details | Review

Description Michael Sheldon 2014-01-20 21:17:28 UTC
Created attachment 266808 [details] [review]
Patch to adjust the weighting of videoconvert's scoring mechanism to preserve colour information

Currently the scoring system used by videoconvert to decide the most appropriate format conversion has a very low value for the loss of colour (a weighting of just 2), whereas the weighting for loss of depth information is considerably higher (a weighting of 32). This can result in the videoconvert element discarding all colour information and using a grayscale format when this format provides a higher bit depth per channel.

For example, when converting I422_10LE into something that videoscale can deal with the current weightings result in GRAY16_BE being selected.

This can be seen in action by executing the following pipeline:

gst-launch-1.0 videotestsrc ! video/x-raw,format=I422_10LE ! videoconvert ! videoscale ! videoconvert ! autovideosink

Here a videotestsrc creates a colour image using the I422_10LE format, however this then gets converted to GRAY16_BE before being processed by the videoscale element, finally it gets converted into YUY2 for display (this may vary depending on chosen sink).

Attached is an example patch which increases the weighting of colour loss over depth loss and so preserves colour information under these circumstances. I'm not sure if a value quite so high as I've provided in the patch is strictly necessary, I'm not especially familiar with the various factors being balanced so it might be a good idea if someone with more experience of videoconvert's scoring system could have a look into the issue.

Perhaps the scoring system should be reworked to look at depth loss over all channels, rather than depth per channel. GRAY16 may have a higher depth per channel than I422_10LE (16 bit vs 10 bit), but it only has one channel, so is discarding far more information than I422_10LE holds overall (and discarding more information than would be kept in a conversion to a 3 channel 8 bit format).

Thanks,
 Mike Sheldon.
Comment 1 Sebastian Dröge (slomo) 2014-01-21 09:17:37 UTC
Review of attachment 266808 [details] [review]:

::: gst/videoconvert/gstvideoconvert.c
@@ +149,3 @@
 #define SCORE_CHROMA_H_LOSS      16
 #define SCORE_DEPTH_LOSS         32
+#define SCORE_COLOR_LOSS         64

COLOR_LOSS is used if we convert from RGB, YUV, GRAY to one of the others. I think RGB<->YUV and GRAY->(RGB|YUV) should get a low score here (like the previous 2). The other conversions should get a very high score like 64 indeed.

Special case is the Y800 and Y16 color format, which basically is like GRAY (and should be treated like GRAY for this case) but within YUV limits.
Comment 2 Wim Taymans 2014-01-24 11:27:10 UTC
Commited a fairly more elaborate and scientifically correct version:

commit 4c8e211e57146f00fff325403c07217274e9b2f1
Author: Wim Taymans <wtaymans@redhat.com>
Date:   Fri Jan 24 12:19:43 2014 +0100

    videoconvert: tweak the scoring algorithm
    
    Make a little table of conversions and manually score them. Use this
    info to define better weights for the scoring algorithm.
    give separate scores for doing changes and the impact of the change,
    This allows us to avoid conversion when we can but still allow fairly
    lossless changes.
    The old code did not penalize GRAY conversions, PAL conversions were
    punished too low and depth conversions too high.
    
    Fixes https://bugzilla.gnome.org/show_bug.cgi?id=722656