After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 754391 - Too much precision shown by default
Too much precision shown by default
Status: RESOLVED FIXED
Product: Gnumeric
Classification: Applications
Component: General
unspecified
Other Linux
: Normal normal
: ---
Assigned To: Jody Goldberg
Jody Goldberg
Depends on:
Blocks:
 
 
Reported: 2015-09-01 12:49 UTC by Ben Gamari
Modified: 2018-05-07 18:24 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Ben Gamari 2015-09-01 12:49:04 UTC
Digits clearly untrustworthy due to floating point rounding error are visible by default in the result of an expression.

Reproduction steps:
  1) Open a new spreadsheet
  2) Enter "=7*0.1" in a cell, press enter
  3) Observe that the result is 0.7000000000000001

This result is of course an unavoidable consequence of using floating point arithmetic. However, arguably the very last digits of a result should not be shown to the user by default. Not only are these digits not accurate, but the user almost certainly doesn't care about the 14th decimal place of their result.

Showing three or four digits is a more appropriate default as this this likely to be the sort of precision that the user desires and should harmlessly conceal unreliable digits in most cases.
Comment 1 Morten Welinder 2018-05-07 18:24:20 UTC
We now limit to 15 significant digits.  That should avoid the worst of the
...000001 and ...999998 nonsense.  The full value is still there and will
be shown if the column is widened.

Note: 3-4 digits would not make anyone happy after entering =1/3.