After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 680475 - rounding issue
rounding issue
Status: RESOLVED DUPLICATE of bug 509965
Product: Gnumeric
Classification: Applications
Component: Analytics
git master
Other Linux
: Normal normal
: ---
Assigned To: Morten Welinder
Jody Goldberg
Depends on:
Blocks:
 
 
Reported: 2012-07-23 18:55 UTC by Frédéric Parrenin
Modified: 2012-07-23 19:08 UTC
See Also:
GNOME target: ---
GNOME version: ---

Attachments
the .gnumeric file to reproduce the problem (3.03 KB, application/x-gnumeric)
2012-07-23 18:55 UTC, Frédéric Parrenin 		Details

Description Frédéric Parrenin 2012-07-23 18:55:39 UTC 
If it is of interest: in the attached sheet, cells in the third row are incremented by 0.1. The result has sometimes only one digit (9, 9.2, 9.5, etc.) and sometimes many digits.
Comment 1 Frédéric Parrenin 2012-07-23 18:55:57 UTC 
Created attachment 219521 [details]
the .gnumeric file to reproduce the problem
Comment 2 Morten Welinder 2012-07-23 19:08:38 UTC 
This comes down to 0.1 not being representable as a finite binary
fractional number.  Set a one-decimal format for the number.

Also, if last-bit precision is important to you, you should avoid long
chains of calculation such as repeatedly adding 0.1.  Rounding errors
may accumulate that way.

The sequence 1/10, 2/10, 3/10, ... is a better choice.

*** This bug has been marked as a duplicate of bug 509965 ***