After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 697293 - =r.psnorm(-0.1,1111,0,1) < 0
=r.psnorm(-0.1,1111,0,1) < 0
Status: RESOLVED FIXED
Product: Gnumeric
Classification: Applications
Component: General
git master
Other All
: Normal normal
: ---
Assigned To: Jody Goldberg
Jody Goldberg
Depends on:
Blocks:
 
 
Reported: 2013-04-04 18:50 UTC by Morten Welinder
Modified: 2013-04-07 20:58 UTC
See Also:
GNOME target: ---
GNOME version: ---

Attachments
sample Gnumeric file (8.04 KB, application/x-gnumeric)
2013-04-05 20:50 UTC, Andreas J. Guelzow 		Details

Description Morten Welinder 2013-04-04 18:50:22 UTC 
=r.psnorm(-0.1,1111,0,1) is not supposed to be -0.01627980883793

And that's a little far for a rounding error.
Comment 1 Morten Welinder 2013-04-04 18:52:32 UTC 
It's really funky there:

-0.1 ---> -0.016280
 0.0 --->  0.437235
 0.1 --->  0.063376

I.e., it is not even close to monotonic.
Comment 2 Morten Welinder 2013-04-05 12:58:15 UTC 
This problem has been fixed in our software repository. The fix will go into the next software release. Thank you for your bug report.
Comment 3 Andreas J. Guelzow 2013-04-05 20:50:21 UTC 
Created attachment 240797 [details]
sample Gnumeric file

I see some curious blips in B17 and B29. Obviously they are well within rounding problem size. I just find it curious that they show up when the surrounding values are listed as 0.
Comment 4 Morten Welinder 2013-04-06 00:50:05 UTC 
There are 18 different parameter areas covering six different evaluation
methods, some used with different parameters.  Minor bumps when moving
from one area to another are to be expected.

I don't think the accuracy claim is better than 10ulp.  I'd have to look
to be sure.
Comment 5 Morten Welinder 2013-04-07 02:32:23 UTC 
On second thought, what we have here is probably a case of catastrophic
cancellation in psnorm.  The reason you see zeros is probably because
I added a clamp to [0;1].
Comment 6 Morten Welinder 2013-04-07 20:58:05 UTC 
Should be fixed now.