GNOME Bugzilla – Bug 558747
Autofill calculates wrong.
Last modified: 2008-11-01 16:11:38 UTC
I fill in: 10 9,6 Mark those two rows and drag an autofill with the result below: 9,2 8,8 8,4 8 7,6 7,2 6,8 6,4 6 5,6 5,2 4,8 4,4 3,99999999999999 3,59999999999999 3,19999999999999 2,79999999999999 Gnumeric 1.8.3 on Gnu/Debian Unstable.
And what is wrong about this? You start with 10 and a number which is approximately 9.6 (9.6 is not representable in base 2). So after you subtracted the values for a while the difference between 9.6 and the true number in that cell accumulates and you get some slightly different number than you would possibly expect. Thanks for the bug report. This particular bug has already been reported into our bug tracking system, but please feel free to report any further bugs you find. (Note that I don't think this is a bug, we are just collecting these reports in 509965.) *** This bug has been marked as a duplicate of 509965 ***
Well, from a users point of view 3,9999999999 was not what I expected. I was trying to enumerate cells with 0,4 steps and 4,4 - 0,4 = 3,99999 was a surprise. Isn't the error in rounding possible to estimate? I thought all calculations in a computer were made in base2, so this rounding error should show almost everywhere why is it visible here? :nisse
In th ecomputer there is no 0.4. Here you are subtracting the same number that is a little bit larger than 0.4 repeatedly. You are not seeing the problem the first few times you are adding only after several repetitions. By this time neither 4.4 nor 0.4 are well represented. This situation doesn't happen very often. And had you formatted that column to reflect that you are expecting only one decimal, then you wouldn't have seen any difference for very much longer. Since differnet numbers have the same representation (approximation) in a comuter there is no way to fix this for all numbers. In fact, any "fix" will just change for which numbers the problem occurs.