After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 748211 - gdk-pixbuf/gdk-pixbuf-xlibrgb.c compilation warning: iteration 128u invokes undefined behavior
gdk-pixbuf/gdk-pixbuf-xlibrgb.c compilation warning: iteration 128u invokes u...
Status: RESOLVED FIXED
Product: gdk-pixbuf
Classification: Platform
Component: general
2.30.x
Other Linux
: Normal normal
: ---
Assigned To: gdk-pixbuf-maint
gdk-pixbuf-maint
Depends on:
Blocks:
 
 
Reported: 2015-04-20 19:48 UTC by Ilya Gordeev
Modified: 2018-04-29 16:44 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Fix the dithering loop and array access (936 bytes, patch)
2017-01-13 22:54 UTC, danny.milo
none Details | Review
gdk-pixbuf-xlib: Fix out-of-bounds error in dithering loop (1.61 KB, patch)
2018-04-28 09:14 UTC, Bert Pauline
none Details | Review
gdk-pixbuf-xlib: Fix out-of-bounds error in dithering loop (1.64 KB, patch)
2018-04-28 10:16 UTC, Bert Pauline
none Details | Review
gdk-pixbuf-xlib: Fix out-of-bounds error in dithering loop (1.64 KB, patch)
2018-04-28 10:24 UTC, Bert Pauline
committed Details | Review

Description Ilya Gordeev 2015-04-20 19:48:12 UTC
Compiling pixbuf-2.30.8 by gcc-4.9.2 I'v got this message:
contrib/gdk-pixbuf-xlib/gdk-pixbuf-xlibrgb.c:1356:16: warning: iteration 128u invokes undefined behavior [-Waggressive-loop-optimizations]
Comment 1 jamiahx 2016-04-10 23:35:59 UTC
it looks like a badly implemented for-loop which gcc is unable to optimize

BTW I got this too with gcc-5.3 and gdk-pixbuf-2.34.0, and its preventing me from installing gdk-pixbuf with the higher security settings I use
Comment 2 danny.milo 2017-01-13 22:54:45 UTC
Created attachment 343460 [details] [review]
Fix the dithering loop and array access
Comment 3 Tobias Mueller 2017-02-17 09:37:17 UTC
yep, looks like a valid change to me.
Using DM[0][i] really seems like a quick hack rather than semantically correct.
Comment 4 Daniel Boles 2017-08-04 16:02:29 UTC
Review of attachment 343460 [details] [review]:

Today in drive-by nit-picking, it's me!

::: gdk-pixbuf-2.36.3/contrib/gdk-pixbuf-xlib/gdk-pixbuf-xlibrgb.c.orig
@@ +1353,3 @@
     {
       DM_565 = malloc(sizeof(guint32) * DM_WIDTH * DM_HEIGHT);
+      for (i = 0; i < DM_HEIGHT; i++)

Add braces around the loop body of this, since now the body spans multiple lines.

I wonder if, since we already have to add new variables, we might take the opportunity to rename them to r/c or x/y for the DM indices and i for DM_565; and move all of these variables' declarations into the if() body.

/restrains self from changing i++ to ++i
Comment 5 Bert Pauline 2018-04-28 09:14:47 UTC
Created attachment 371486 [details] [review]
gdk-pixbuf-xlib: Fix out-of-bounds error in dithering loop

Here is an updated version of the original patch by danny.milo@[...] with the suggestions of Daniel Boles incorporated.
Comment 6 Emmanuele Bassi (:ebassi) 2018-04-28 09:37:19 UTC
Review of attachment 371486 [details] [review]:

Looks good to me, with a small nitpick.

::: contrib/gdk-pixbuf-xlib/gdk-pixbuf-xlibrgb.c
@@ +1356,2 @@
 	{
+	  for (x = 0; x < DM_WIDTH; x++, i++)

You're not initializing `i` to 0, which means `i++` could do anything — especially in aggressively optimized builds.

I also find it slightly less readable, and we don't use this pattern often in GLib/GTK/GdkPixbuf.

I'd rather have the `i` incremented explicitly inside the inner loop, e.g.:

```
DM_565[i++] = (dith << 20) | dith | (((7 - dith) >> 1) << 10);
```

After initializing `i` either when declaring it, or before entering the outer for loop.
Comment 7 Bert Pauline 2018-04-28 10:16:26 UTC
Created attachment 371488 [details] [review]
gdk-pixbuf-xlib: Fix out-of-bounds error in dithering loop

Hi Emmanuele

Thanks for the review.

> Looks good to me, with a small nitpick. [...] You're not initializing `i` to 0.

That's not a small nitpick. That's a big oversight on my side.
I fixed that and explicitly increment i in the loop.
Comment 8 Emmanuele Bassi (:ebassi) 2018-04-28 10:24:27 UTC
Review of attachment 371488 [details] [review]:

Looks good, thanks for the quick fix!
Comment 9 Bert Pauline 2018-04-28 10:24:32 UTC
Created attachment 371489 [details] [review]
gdk-pixbuf-xlib: Fix out-of-bounds error in dithering loop

Sorry, there was an oversight in the patch. The variable i has to be incremented after the print statement.
Comment 10 Emmanuele Bassi (:ebassi) 2018-04-28 10:25:17 UTC
Review of attachment 371489 [details] [review]:

Nice :thumbsup:
Comment 11 Emmanuele Bassi (:ebassi) 2018-04-29 16:44:41 UTC
Attachment 371489 [details] pushed as 979a87b - gdk-pixbuf-xlib: Fix out-of-bounds error in dithering loop