After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 793900 - 2.7.5: test failure: 105 - python-bindings (regression from 2.7.3 ??)
2.7.5: test failure: 105 - python-bindings (regression from 2.7.3 ??)
Status: RESOLVED FIXED
Product: GnuCash
Classification: Other
Component: Build system
2.7.x
Other Linux
: Normal normal
: ---
Assigned To: gnucash-core-maint
gnucash-core-maint
Depends on:
Blocks:
 
 
Reported: 2018-02-27 21:41 UTC by Dmitry Smirnov
Modified: 2018-06-30 00:05 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Dmitry Smirnov 2018-02-27 21:41:47 UTC
On "amd64" on Debian unstable "python-bindings" tests fails as follows:

~~~~
The following tests FAILED: 
        105 - python-bindings (Failed) 
Errors while running CTest 
make[5] *** [CMakeFiles/check.dir/build.make:71: CMakeFiles/check] Error 8 
make[5] Leaving directory '/build/gnucash-2.7.4/.build' 
make[4] *** [CMakeFiles/Makefile2:208: CMakeFiles/check.dir/all] Error 2 
make[4] Leaving directory '/build/gnucash-2.7.4/.build' 
make[3] *** [CMakeFiles/Makefile2:215: CMakeFiles/check.dir/rule] Error 2 
make[3] Leaving directory '/build/gnucash-2.7.4/.build' 
make[2] *** [Makefile:213: check] Error 2 
make[2] Leaving directory '/build/gnucash-2.7.4/.build' 
~~~~

The following is from LastTest.log:

~~~~
105/105 Testing: python-bindings
105/105 Test: python-bindings
Command: "/usr/bin/python" "/build/gnucash-2.7.4/bindings/python/tests/runTests.py.in"
Directory: /build/gnucash-2.7.4/.build/bindings/python/tests
"python-bindings" start time: Feb 27 21:15 UTC
Output:
----------------------------------------------------------
* 21:15:16  WARN <gnc.core-utils> no backend loaded, or the backend doesn't define register_cb, returning 0
* 21:15:16  WARN <gnc.core-utils> no backend loaded, or the backend doesn't define register_cb, returning 0
* 21:15:16  WARN <gnc.core-utils> no backend loaded, or the backend doesn't define register_cb, returning 0
* 21:15:16  WARN <gnc.core-utils> no backend loaded, or the backend doesn't define register_cb, returning 0
* 21:15:16  WARN <gnc.core-utils> no backend loaded, or the backend doesn't define register_cb, returning 0
Traceback (most recent call last):
  • File "/build/gnucash-2.7.4/bindings/python/tests/runTests.py.in", line 12 in <module>
    from test_split import TestSplit
  • File "/build/gnucash-2.7.4/bindings/python/tests/test_split.py", line 4 in <module>
    from unittest_support import *
ImportError: No module named unittest_support
<end of output>
Test time =   0.15 sec
~~~~
Comment 1 John Ralls 2018-03-05 17:03:36 UTC
This should be fixed already in 2.7.5, released 25 Feb, which is what you should be testing against if you insist on testing tarballs instead of from git.
Comment 2 Dmitry Smirnov 2018-03-05 22:50:35 UTC
Thanks, I did not realise that 2.7.5 is ready. I'll try to build id ASAP.

FYI sourceforge had some problems and may not have been properly updated when I was downloading 2.7.4 -- it took me several attempts to get the tarball and I was only able to download it on the following day. I've used our usual "uscan" script to download the latest version and only 2.7.4 was available at a time...
Comment 3 John Ralls 2018-03-05 23:31:18 UTC
Hmm. It's there and I just downloaded it manually, but they have been having issues the last couple of weeks. Maybe you should follow the releases on Github.
Comment 4 Dmitry Smirnov 2018-03-06 21:13:19 UTC
Tracking github as well might be a good idea as it is very stable...

I tried 2.7.5 but it failed this test exactly the same...
Comment 5 John Ralls 2018-03-07 03:48:18 UTC
Python bindings test from the current unstable HEAD works for me on a freshly-updated Debian Sid and-64. You'll have to dig into it a bit more yourself.
Comment 6 Dmitry Smirnov 2018-03-07 12:11:28 UTC
I'm not sure I'm capable of finding the problem... I already tried to compare with 2.7.3 but couldn't identify the culprit... Due to other commitments I can only spare very little time for thins, not nearly enough and I have no one else to ask for help... 2.7.3 works.
Comment 7 John Ralls 2018-03-07 14:39:35 UTC
Can you provide or point to instructions for creating the exact environment you're building in?
Comment 8 Dmitry Smirnov 2018-03-07 22:00:54 UTC
I use "pbuilder" tool to build in pristine "unstable". It would be more or less equivalent of environment you get using "debootstrap". Build system installs all packages from Build-Depends and that's pretty much all that is done to make disposable build environment that is used exactly once for a build.

I have to run now but I can provide the exact build commands later if you wish...

Thank you.
Comment 9 John Ralls 2018-03-08 17:10:29 UTC
Yes, please provide the exact commands you use, including the pbuilder setup.
If you use a script, please attach it. (If you don't you should probably write one!)
Comment 10 Dmitry Smirnov 2018-03-08 21:36:11 UTC
If you have Debian "unstable" VM then you can just add "debian" folder from [1] to Gnucash source tree and invoke the following command provided by "devscripts" package:

    debuild -us -uc -b

Pbuilder is easy enough:

    sudo pbuilder --create       ## prepare "sid" tarball/container
    sudo pbuilder --update       ## update build environment packages

then from source tree

    pdebuild

[1]: https://anonscm.debian.org/cgit/pkg-gnucash/gnucash.git?h=experimental

That's pretty much it, only standard build tools are used and no custom scripts are necessary. Technically speaking scripts are in "debian/rules" file which is actually a Makefile...
Comment 11 John Ralls 2018-03-09 22:32:18 UTC
Thanks, I found the problem. It turned out to be a tarball issue which is why I couldn't reproduce it at first. It's fixed and I've fiddled the distcheck target to build with Python enabled so such problems will get detected at release instead of later.

A couple of notes on pkg-gnucash: In control, libgoffice-0.10 isn't required, we've pulled the tiny bit of code we needed from there into gnucash. Geert removed the intltool requirement last week, it's now pure gettext so that can go, too. In rules, you should remove the BUILDING_FROM_VCS hack. There is no autogen.sh anymore so it breaks building from git and the code in utils/gnc-vcs-info works fine.
Comment 12 Geert Janssens 2018-03-09 22:45:05 UTC
(In reply to John Ralls from comment #11)
> In rules, you should remove the
> BUILDING_FROM_VCS hack. There is no autogen.sh any more so it breaks building
> from git and the code in utils/gnc-vcs-info works fine.

I'm not too sure about this particular piece of advice. I seem to remember the BUILDING_FROM_VCS hack was introduced because downstream packagers can also store our tarball in git in which case gnc-vcs-info draws the wrong conclusion.

You're certainly right the autogen.sh test will no longer work. Perhaps it should be replaced with an existence test on libgnucash/core-utils/gnc-vcs-info.h. If that file exists, the build is from a tarball, otherwise it's a source repo based build.
Comment 13 John Ralls 2018-03-09 23:07:05 UTC
There's also the contrary issue of people trying to use Github-generated tarballs, in which case gnc-vcs-info gives the wrong answer in the other direction.

Another consideration: The only reason for wrapping a tarball into a vcs repo is because you want to change it, and in that case you may need to re-swig it. Not only that, but once it's changed then the contents of gnc-vcs-info.h should change too. It's also possible to use a vcs other than git, svn, or svk, or to change the code without using a vcs at all.
Comment 14 Geert Janssens 2018-03-10 10:29:32 UTC
(In reply to John Ralls from comment #13)
> There's also the contrary issue of people trying to use Github-generated
> tarballs, in which case gnc-vcs-info gives the wrong answer in the other
> direction.

In that case the build will fail because the extracted tarball isn't a git repository so the build system assumes certain files to be there that are in fact missing: gnc-version.h (which we really should eliminate as it's become a static file), gnc-version-info.h, ChangeLog, gnucash manpage, gnucash.pot.

> Another consideration: The only reason for wrapping a tarball into a vcs
> repo is because you want to change it, and in that case you may need to
> re-swig it.

Not necessarily. Distributions also do this to guarantee reproducibility. The store the exact source they have used to build a certain package.
It may be altered indeed, but that's usually with patches kept in a separate tree.

Regardless, I have recently altered the build system to separate swig generation from the repository presence detection. There are now two environment variables to control this:
GENERATE_SWIG_WRAPPERS can be used to override whether or not to generate the wrappers. If not set it will fall back to the previous default being: only generate the wrappers when building from vcs.
BUILDING_FROM_VCS controls generation of any file that depends on git history to be available (ChangeLog, gnc-version-info.h)
So setting GENERATE_SWIG_WRAPPERS to yes will cover this situation.

> Not only that, but once it's changed then the contents of
> gnc-vcs-info.h should change too. It's also possible to use a vcs other than
> git, svn, or svk, or to change the code without using a vcs at all.

True. First off, anything other than git will not work properly any more. In particular the ChangeLog generation is hard wired to query git history and gnucash will report git as vcs to the user regardless of what the real vcs was when generating the binaries. In that context I think it's probably time to drop support for the other version control systems from utils/gnc-vsc-info. It made sense in the transition period from svn to git, I doubt it still makes sense now.

Having said that I agree we would prefer the version info to be altered if the sources are changed outside (a clone of) our git repository. Unfortunately that's something we can only recommend, not enforce. There are currently two ways to handle this:
1. updating gnc-vcs-info.h in the source tree
2. set GNUCASH_BUILD_ID to override the version information we otherwise would extract from gnc-vcs-info.h. While writing this I believe we could still polish the experience from this.

Lastly, one could argue it would also be preferable to update the changelog when making changes outside  (a clone of) our git repository.
Comment 15 John Ralls 2018-03-10 17:01:09 UTC
This bug report isn't really the right place to discuss this, let's move to the development list.
Comment 16 Dmitry Smirnov 2018-03-11 10:31:24 UTC
Thanks for the fix and for feedback.

"intltool" is actually still required on 2.7.5 (CMake Error at CMakeLists.txt:256 (MESSAGE): Can't find intltool-merge program.  Please set INTLTOOL_MERGE.) as Gnucash FTBFS later if "intltool" is not present...

In the past I've used "BUILDING_FROM_VCS" as it's not the first time I've heard "works in git" when tarball FTBFS... ;)  I wish I could check for "libgnucash/core-utils/gnc-vcs-info.h" but it is present in the 2.7.5 tarball...

As advised I've removed "libgoffice-0.10-dev" from Build-Depends (I've forgotten about it)... Thanks.
Comment 17 John Ralls 2018-03-11 14:56:41 UTC
Intltool is removed in 2.7.6, which I'll release today.

Right, gnc-vcs-info.h is present in the tarball and is generated in a git build, so you'd reverse the logic from autogen.sh for overriding BUILDING_FROM_VCS. Geert's away for 2 weeks, there are still a few significant bugs to fix, and we want to get 3.0 wrapped up by the end of the month so we'll more than likely leave fixing BUILDING_FROM_VCS (see https://lists.gnucash.org/pipermail/gnucash-devel/2018-March/041733.html) for 3.1.
Comment 18 John Ralls 2018-06-30 00:05:01 UTC
GnuCash bug tracking has moved to a new Bugzilla host. This bug has been copied to https://bugs.gnucash.org/show_bug.cgi?id=793900. Please update any external references or bookmarks.