After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 794933 - SQL woes: Backend not found when opening/storing data file and converted XML file crashing GnuCash
SQL woes: Backend not found when opening/storing data file and converted XML...
Status: RESOLVED FIXED
Product: GnuCash
Classification: Other
Component: Backend - SQL
3.0
Other Mac OS
: Normal normal
: future
Assigned To: gnucash-core-maint
gnucash-core-maint
: 795593 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2018-04-03 14:32 UTC by Jo Wetzig
Modified: 2018-06-30 00:06 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
MacOS crash report from opening converted XML with 3.0 (66.11 KB, text/plain)
2018-04-03 15:43 UTC, Adrien
Details
Crash Report on Macbook trying to open an xml file (76.38 KB, text/plain)
2018-04-03 23:45 UTC, Jo Wetzig
Details
Partial copy of crashing data file (13.04 KB, text/plain)
2018-04-03 23:46 UTC, Jo Wetzig
Details

Description Jo Wetzig 2018-04-03 14:32:47 UTC
On Mac OS 10.12.6 (Sierra, build 16G1314) Versions 2.77 and 3.0 of GnuCash do 
"not find a suitable backend" 
when attempting to open a sqlite3 style data file (named: 'gnusql.gnucash', created by V. 2.6.19). Nor do they present SQL as a possible storage format, when attempting to store a newly created --mostly empty-- data file. The only storage format offered is "xml".

I have verified, that 
/Applications/Gnucash.app/Contents/Resources/lib/dbd contains
   libdbdmysql.so   libdbdpgsql.so  and libdbdsqlite3.so
and that 
   libgnc-backend-sql.dylib
is present in 
/Applications/Gnucash.app/Contents/Resources/lib/

The environment file is unchanged from the distribution bundle.
While running GnuCash V3.0 the path displayed in a parallel Terminal Window is
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Server.app/Contents/ServerRoot/usr/bin:/Applications/Server.app/Contents/ServerRoot/usr/sbin
(though in a Mac the libraries would be all in the application bundle, I presume)


The V 2.6.19 data file, when converted under V 2.6.19 by "save as" XML into a XML file crashes V 2.7.7 and V 3.0 with an
'Application Specific Information:
terminating with uncaught exception of type boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::bad_lexical_cast> >: bad lexical cast: source type value could not be interpreted as target
abort() called'
error.
Comment 1 John Ralls 2018-04-03 15:17:43 UTC
For the backend issue, try editing Gnucash.app/Contents/Resources/etc/gnucash/environment, adding
  GNC_DBD_DIR={SYS_LIB}/dbd
at the end.

$PATH in your terminal is whatever you set it up to be in the shell's profile and rc files. While a program run from a terminal prompt might change the environment (GnuCash does, via the environment file, though it doesn't do anything with PATH) those changes will affect only that instance of the program and will be invisible to the shell.
That aside, only Microsoft Windows uses the path to find shared libraries.

The crash sounds very much like bug 782144 and bug 793768. Please attach the crash report. If you can help isolate the item that produces the crash it may help on the other two bugs as well.
Comment 2 Adrien 2018-04-03 15:43:05 UTC
Same issue here on High Sierra.

However, for me, saving the file as XML using 2.6.19 and opening in 3.0 crashed the first time, but was successful after that. The 'no suitable backend' still appeared if trying to open the sqlite version.

(In reply to John Ralls from comment #1)
> For the backend issue, try editing
> Gnucash.app/Contents/Resources/etc/gnucash/environment, adding
>   GNC_DBD_DIR={SYS_LIB}/dbd
> at the end.

John, this worked for me at least. The sqlite file now loads without issue. Thanks.

> The crash sounds very much like bug 782144 and bug 793768. Please attach the
> crash report. If you can help isolate the item that produces the crash it
> may help on the other two bugs as well.

Thought it may not be the same issue, I've attached my crash report on that re-open with 3.0 after conversion using 2.6.19. (as noted, it only crashed once, subsequent opens were successful)
Comment 3 Adrien 2018-04-03 15:43:53 UTC
Created attachment 370490 [details]
MacOS crash report from opening converted XML with 3.0
Comment 4 Jo Wetzig 2018-04-03 23:45:50 UTC
Created attachment 370506 [details]
Crash Report on Macbook trying to open an xml file
Comment 5 Jo Wetzig 2018-04-03 23:46:49 UTC
Created attachment 370507 [details]
Partial copy of crashing data file
Comment 6 Jo Wetzig 2018-04-04 00:11:15 UTC
Hello John

many thanks for the hint on the environment. My major trouble is gone, the bug might be considered "solved".

I'll just keep it open though with the second part, the crashing xml file (when will I learn not to do double bugging in one message?;))).

I have just attached the crash report and a partial copy (nose and tail) of the crashing data file.
Please note that I have replaced confidential data --which was only alphanumeric ascii-- in the data file by "***intentionally hidden***".

(Please note also, that the sql file which was converted to xml had some non-standard tax information. I fudged txf.scm, txf-help.scm and taxtxf.scm to contain categories for German income tax ("Einkommenssteuer") and linked the accounts to those tax categories in the edit menu. I believe, since I just added infos to the scheme files, that the program logic flow and the logical organization of the data file was not changed relevantly.)

Greetings from Germany

J W
Comment 7 Frank H. Ellenberger 2018-04-04 18:02:12 UTC
OT: Jo if your "Einkommenssteuer" files follow the official forms, would you like to share them?
Comment 8 John Ralls 2018-04-15 19:19:19 UTC
Sorry for the delay. It's been a busy couple of weeks (and no sign of that changing...)

The first crash (Adrien's) looks like it has nothing to do with the data, which seems to have successfully loaded. Instead it seems that Gdk was trying to handle a mouse event on a toplevel window that had been freed. Unless you can reliably reproduce the crash I don't think I can debug it enough to fix it.

The prime suspect for the unhandled exception in the second (Jo's) crash report is a bad date and Jo carefully removed all of the dates from the file so unfortunately there's nothing to look for. The modified file isn't valid XML so it naturally won't load.
Comment 9 Arnd Hostert 2018-04-16 13:43:32 UTC
I had the same crash reason.

It turned out to be a malformed date:

426139:    <ts:date>13-04-26 11:52:28 +0053</ts:date>

I found the corresponding transaction in 2.6.21, changed the date using the date picker and saved the file.

It loaded into 3.0 without problems.


I too would be interested in the Einkommensteuer part ......
Comment 10 Frank H. Ellenberger 2018-04-16 18:55:27 UTC
(In reply to Arnd Hostert from comment #9)
> I too would be interested in the Einkommensteuer part ......

Follow
https://lists.gnucash.org/pipermail/gnucash-de/2018-April/010266.html
Comment 11 John Ralls 2018-04-16 23:20:14 UTC
There's a nightly build at https://wiki.gnucash.org/builds/win32/maint/gnucash-3.0-2018-04-16-git-3.0-75-g87f94abc8+.setup.exe that includes a change to the date parser that should prevent the crash if it's due to a malformed date. Please test.
Comment 12 Geert Janssens 2018-04-17 12:39:53 UTC
The nightly build in the comment above won't run due to a missing dll. I have just now created a new installer to fix that:
https://wiki.gnucash.org/builds/win32/maint/gnucash-3.0-2018-04-17-git-3.0-75-g87f94abc8+.setup.exe
Please test with that one and report back.
Comment 13 Jo Wetzig 2018-04-18 05:13:07 UTC
Sorry, Mac only :(
Comment 14 Arnd Hostert 2018-04-18 13:45:57 UTC
Me too. Mac only.....
Comment 15 John Ralls 2018-04-27 02:50:55 UTC
*** Bug 795593 has been marked as a duplicate of this bug. ***
Comment 16 John Ralls 2018-06-07 20:59:16 UTC
The fix was in GnuCash 3.1, released 28 April. Is anyone having this problem with 3.1?
Comment 17 matthew 2018-06-07 21:29:11 UTC
I opened Bug 795593. 3.1 has been working great for me on MacOS. Really like the addition of mysql drivers with the release. Switched everything from local sqlite files to mysql on linux and very happy now with everything using a central database (have other read only apps that use the gnucash database for generating reports and integrating with other systems).
Comment 18 Jo Wetzig 2018-06-17 13:52:41 UTC
GC 3.1 
Created a new data file, format sqlite3.
Added one transaction
saved
reopened OK
saved as xml
reopened xml OK

IMHO the bug is fixed and can be closed :)

Cheers

JW 

BTW: Status for me shows as "NEEDINFO", however resolved as "FIXED". In my (un-English) understanding this is a contradiction.

Puzzled ;)
JW
Comment 19 John Ralls 2018-06-17 14:03:17 UTC
Dunno where you were seeing it resolved/fixed, but it is now.
Comment 20 John Ralls 2018-06-30 00:06:48 UTC
GnuCash bug tracking has moved to a new Bugzilla host. This bug has been copied to https://bugs.gnucash.org/show_bug.cgi?id=794933. Please update any external references or bookmarks.