After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 746247 - div255 implementation is incorrect
div255 implementation is incorrect
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: orc
git master
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2015-03-15 13:40 UTC by Jan Schmidt
Modified: 2018-11-03 10:47 UTC
See Also:
GNOME target: ---
GNOME version: ---

Attachments
test div255 app (256 bytes, text/x-python)
2015-03-15 13:40 UTC, Jan Schmidt 		  Details
Fix div255w implementation (4.86 KB, patch)
2015-03-15 13:41 UTC, Jan Schmidt 		none Details | Review
Update tables in docs (37.72 KB, patch)
2015-03-15 13:41 UTC, Jan Schmidt 		none Details | Review

Description Jan Schmidt 2015-03-15 13:40:48 UTC 
Created attachment 299452 [details]
test div255 app

div255 implements this algorithm everywhere:
  d = (s + 128 + (s+128)>>8) >> 8
which produces a result that is off-by-one for roughly half the 0..65535 input range.

A correct implementation is:
  d = (s + 1 + (s >> 8)) >> 8

Test python app, and a fix which implements the new algorithm attached.
Comment 1 Jan Schmidt 2015-03-15 13:41:26 UTC 
Created attachment 299453 [details] [review]
Fix div255w implementation

The current implementation of div255w produces incorrect
off-by-one values for a large number of values in the input
range. Use a better bit-shifting implementation that gets it
right, and document that it doesn't work for 0xffff
Comment 2 Jan Schmidt 2015-03-15 13:41:36 UTC 
Created attachment 299454 [details] [review]
Update tables in docs
Comment 3 Jan Schmidt 2015-03-15 13:45:18 UTC 
I modified most of the SIMD implementations, but I can only verify SSE, and I don't know how to modify the NEON implementation.
Comment 4 Jan Schmidt 2015-03-15 14:35:05 UTC 
I think NEON needs to do:

tmpc = orc_compiler_get_constant (1)
vshr.u16 tmp, src, 8
vadd.i16 tmp, src
vaddhn.i16 dest, tmp, tmpc

but I'm not sure how to encode that.
Comment 5 Sebastian Dröge (slomo) 2018-07-23 08:33:21 UTC 
See also https://bugzilla.gnome.org/show_bug.cgi?id=796846
Comment 6 GStreamer system administrator 2018-11-03 10:47:44 UTC 
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/orc/issues/7.