After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 771617 - Build fails because test test-report-utilities is sensitive to time of day/timezone
Build fails because test test-report-utilities is sensitive to time of day/ti...
Status: RESOLVED FIXED
Product: GnuCash
Classification: Other
Component: Build system
2.6.14
Other Linux
: Normal normal
: ---
Assigned To: gnucash-core-maint
gnucash-core-maint
Depends on:
Blocks:
 
 
Reported: 2016-09-18 11:28 UTC by Christoph Korn
Modified: 2018-06-29 23:51 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Christoph Korn 2016-09-18 11:28:04 UTC
Build a Debian package for 2.6.14 there is a build error because of a test:
test-report-utilities

This is the build log:
https://paste.ubuntuusers.de/423128/

And this is the content of the file:
src/report/report-system/test/test-suite.log

https://paste.ubuntuusers.de/423133/

The missing files exists here. Maybe it is searched in the wrong directory:
(xenial-amd64)korn@pc:~/packages/gnucash/gnucash-2.6.14$ find . -iname "*gncmod-backend-dbi*"
./src/backend/dbi/.libs/libgncmod-backend-dbi.la
./src/backend/dbi/.libs/libgncmod-backend-dbi.so
./src/backend/dbi/.libs/libgncmod-backend-dbi.lai
./src/backend/dbi/libgncmod-backend-dbi.la

Building takes place in a chroot environment with Ubuntu 16.04 inside.
Comment 1 John Ralls 2016-09-18 13:49:23 UTC
This looks like the failure we get intermittently on Travis, which also uses Ubuntu, though 14.04 instead of 16.04.
Comment 2 Geert Janssens 2016-10-03 10:05:35 UTC
I have come across this bug yesterday again and investigated some more. It turns out the result of test-report-utilities.scm is time-dependent due to a combination of factors.

1. https://github.com/Gnucash/gnucash/blob/maint/src/report/report-system/test/test-report-utilities.scm#L21
end-date in this line is set starting with current-time, which returns a time of day. Nowhere in the rest of the code this time information is normalized.

2. https://github.com/Gnucash/gnucash/blob/maint/src/report/report-system/test/test-report-utilities.scm#L29
This line creates a set of transactions with dates inferred from the end-date above (10 transactions, first transaction 9 days before end-date, last transaction on end-date). Creating a transaction does normalize the time information to 11:00:00 UTC

3. https://github.com/Gnucash/gnucash/blob/maint/src/report/report-system/test/test-report-utilities.scm#L31
gnc:account-get-trans-type-splits-interval runs a qof query to find the splits in the given date interval, starting 4 days before end-date and ending on end-date. I don't think qof queries run further normalizations on the given dates, so it now depends on whether end-date had time information before or after 11:00:00 UTC whether 10 or 8 splits are found.

I tried a few approaches to fixing this, but wasn't really successful.

I first added a few lines to normalize the time information in the value returned by (localtime (current-time)) in line 21 with this diff:
--- a/src/report/report-system/test/test-report-utilities.scm                                                       
+++ b/src/report/report-system/test/test-report-utilities.scm                                                       
@@ -18,8 +18,14 @@                                                                                                  
                                                                                                                    
 (define (test-account-get-trans-type-splits-interval)                                                              
   (let ((env (create-test-env))
-       (end-date (gnc:date->timepair (localtime (current-time)))))
-    (let* ((accounts (env-create-account-structure-alist env (list "Assets"
+       (end-time (gmtime (current-time))))
+       (set-tm:sec end-time 0)
+       (set-tm:min end-time 0)
+       (set-tm:hour end-time 11)
+       (set-tm:gmtoff end-time 0)
+       (set-tm:zone end-time "UTC")
+    (let* ((end-date (gnc:date->timepair end-time))
+           (accounts (env-create-account-structure-alist env (list "Assets"
                                                                   (list (cons 'type ACCT-TYPE-ASSET))
                                                                   (list "Bank Account")
                                                                   (list "Wallet"))))

This did slightly improve the sensitivity to time changes. However this code is still sensitive to timezone changes. I could make it work by setting environment variable TZ to "PST+6" (don't know if PST is really +6 but the name doesn't matter here), and it would still fail for GMT+0.

That's probably because gnc:date->timepair in line 21 calls guile's mktime which is also timezone sensitive.

I'm not sure what the best way out would be. Perhaps these guile date/time related functions should be rewritten to use our qof implementation of date and time, or perhaps it's just a few small changes I don't see to ensure our guile routines make the same date/time assumptions as the qof routines.
Comment 3 John Ralls 2016-10-03 14:37:09 UTC
We absolutely should use gnc-date instead of built-in Guile functions if for no other reason than that Guile very likely uses time_t which is still an int, even in 64-bit builds, in Darwin and perhaps other BSD systems. That's probably out-of-scope for this bug.

The place to do the normalization is in test-account-get-trans-type-splits-interval. The start date, presently just calculated with (decdate end-date (NDayDelta 5)), needs to be set to 00:00:01 and the end-date to 23:59:59. The test will still fail in +14 and +15, but that's the compromise we made when we changed all transactions to happen at 11:00Z.
Comment 4 Geert Janssens 2016-10-03 15:48:39 UTC
Thanks for the clarification John. I have altered the test to use an interval as you suggested and indeed it works for every timezone except for +11.

As for your other comment about replacing our use of guile's date/time functions with gnc-date, I have opened bug 772369 to track this.
Comment 5 Christoph Korn 2016-10-03 17:05:37 UTC
What is the commit that fixes it, so I can backport it for my build?
Comment 6 John Ralls 2016-10-03 17:12:24 UTC
 c605e44, which also happens to be maint HEAD until someone pushes something else.
Comment 7 Christoph Korn 2016-10-03 17:14:12 UTC
Ah, was looking at the wrong branch. Thanks.
Comment 8 Dmitry Smirnov 2016-12-12 21:17:09 UTC
I'm experiencing build failures as well and I'm struggling to backport fixes to 2.6.14... Is there plans to accommodate fixes to new release soon-ish?
Comment 9 John Ralls 2016-12-12 21:27:41 UTC
(In reply to Dmitry Smirnov from comment #8)
> I'm experiencing build failures as well and I'm struggling to backport fixes
> to 2.6.14... Is there plans to accommodate fixes to new release soon-ish?

http://wiki.gnucash.org/wiki/Release_Schedule

That aside don't backport fixes, just clone the repo and use the maint branch.
Comment 10 Dmitry Smirnov 2016-12-12 23:07:18 UTC
Thanks.

> just clone the repo and use the maint branch.

Unfortunately that won't be suitable for official Debian package...

It would be easier to wait for 2.6.15 if its release is planned for only about a week from now...
Comment 11 John Ralls 2016-12-12 23:15:34 UTC
We would prefer that you not patch our release tarballs for official distribution, it can make it difficult to replicate bug reports.

A far better process would be for you to do frequent builds of the maint branch and immediately report any bugs you find so that the release tarballs will be clean for your package.
Comment 12 Dmitry Smirnov 2016-12-23 08:47:10 UTC
That would be unusual and more demanding work flow.  Due to lack of time I may not be able to check pre-releases from "maint" branch. To do what you ask I need to have some sort of CI process... Typically my attention is triggered by new release which I test, build and upload if everything is all right. Regularly re-building "maint" branch could be 10 times more work...
Comment 13 John Ralls 2016-12-23 15:00:58 UTC
ISTM that patching tarballs is more demanding than running tests. What sort of tests are you doing that are so much work?
 
We already have a CI solution set up at https://travis-ci.org/Gnucash/gnucash. It uses Ubuntu 14.04, which is the same as Debian Jessie as far as GnuCash's test suite is concerned. If you want more runs or have additional tests, for example of different time zones, we should be able to work something out. We don't have any GUI tests in our test suite, though, and don't know how to automate them, so a CI approach won't work for that.
Comment 14 John Ralls 2018-06-29 23:51:11 UTC
GnuCash bug tracking has moved to a new Bugzilla host. This bug has been copied to https://bugs.gnucash.org/show_bug.cgi?id=771617. Please update any external references or bookmarks.