After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 781634 - alert on missing account currency when opening account (since GC2.6.16)
alert on missing account currency when opening account (since GC2.6.16)
Status: RESOLVED OBSOLETE
Product: GnuCash
Classification: Other
Component: Currency and Commodity
2.6.16
Other All
: Normal major
: ---
Assigned To: gnucash-core-maint
gnucash-core-maint
: 783285 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2017-04-23 15:10 UTC by Jannick
Modified: 2018-06-29 23:56 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Jannick 2017-04-23 15:10:36 UTC
Starting with GC 2.6.16 on Win10 (German version) GC fires up an alert to be unable to find the account curreny and using the system currency instead when opening an account and even when opening a GC file.

German version of the alert text: 'Kontowährung konnte nicht bestimmt werden. Stattdessen wird die voreingestellte Systemwährung verwendet.'

That this appears on Win OS / German version - while there are no other complaints on the GC user list as of now - suggests that this could well be a locale issue.

Changing the general currency config (locale vs explicit choice) does not change this behaviour. However, the issues disappear when downgrading to 2.6.15 (not desirable due to other bug fixes in 2.6.16).


Separately observations/new issues after upgrading to 2.6.15 (from 2.6.15) ... hoping that these information could be helpful:

1 - it appears that in the account view the association of account currency / book currency to the displayed totals were changed   
2 - in the views (PnL, b/s, accounts) the amounts in account currency are additionaly displayed by default; moreover, these views require for a pure EUR book a FXR for EUR/EUR of 1 to avoid that zero (sub-)totals in book currency are wrongly shown.
3 - Alert misleading since in the xml structure the account currency is available for checked accounts, e.g.

<gnc:account version="2.0.0">
  ...
  <act:commodity>
    <cmdty:space>CURRENCY</cmdty:space>
    <cmdty:id>EUR</cmdty:id>
  </act:commodity>
  ...
</gnc:account>
Comment 1 Geert Janssens 2017-04-23 17:34:24 UTC
Thanks for your report.

I can't reproduce on my test system with the limited testing I've done.

There are a couple of differences with your environment:
1. I'm on Windows XP. Maybe it's something Windows 10 only
2. My locale is Belgian/Dutch. I've also tried setting LANG and LANGUAGE to de__DE in the gnucash environment file to simulate a German set up. But I still don't get warnings.
3. My tests may be fake, because I also have a development environment for gnucash installed on the same PC and perhaps the officially installed gnucash still finds missing files in there.

I'm mostly adding this as extra data points in hopes to reduce the possible causes. Nu solution yet unfortunately.
Comment 2 Frank H. Ellenberger 2017-04-23 17:40:28 UTC
Just for completeness:
This was before on the mailing lists reported in following threads:

http://lists.gnucash.org/pipermail/gnucash-de/2017-April/009887.html
http://lists.gnucash.org/pipermail/gnucash-user/2017-April/070266.html
Comment 3 hhn10 2017-04-24 09:22:58 UTC
The same issue happens on my windows 10 machine. My regional settings are set to Swiss German. For each account tab that opens I get a message that the currency can't be determined. More observations:

When entering a transaction from e.g. a bank account to an expense, I am asked to enter a rate from CHF to CHF..

When trying to enter the sell of a fund by simply entering the number and total, the balance of the account where the money went didn't get updated.. 

at this time I downgraded to 2.6.15 :-)
Comment 4 hhn10 2017-04-24 18:14:30 UTC
I just reinstalled 2.6.16 and set the two lines in the environment file to 
LANG=en_US
LANGUAGE={LANG}
as described in the wiki - I don't get the error messages anymore and also the sell transaction could be entered as expected.
Comment 5 Jannick 2017-04-24 21:22:44 UTC
Thanks - great idea: switching the language to 'en_US' blows away all the issues away here on my Win10 machine (German version).
Comment 6 Geert Janssens 2017-04-24 21:59:06 UTC
A few more bits of information to help me or another dev onwards in debugging:
The English version of the German error message is:
"Could not determine the account currency. Using the default currency "
"provided by your system."

This message is used in
src/register/ledger-core/split-register-load.c:277

Just above that is a call to gnc_account_or_default_currency. It looks like that function returns an unexpected result. Don't know why that is though as I can't reproduce the issue.
Comment 7 Jannick 2017-04-25 20:49:27 UTC
Manually replacing the cmdty:space element from CURRENCY to ISO4217 in the xml file (see example below) solves the issue on my Win10 box.

This could probably explain the behaviour in the function 'gnc_account_or_default_currency' (#c6) (which I have not traced down).

1 - Before change:

<gnc:account version="2.0.0">
  ...
  <act:commodity>
    <cmdty:space>CURRENCY</cmdty:space>
    <cmdty:id>EUR</cmdty:id>
  </act:commodity>
  ...
</gnc:account>

2 - After change:

<gnc:account version="2.0.0">
  ...
  <act:commodity>
    <cmdty:space>ISO4217</cmdty:space>
    <cmdty:id>EUR</cmdty:id>
  </act:commodity>
  ...
</gnc:account>
Comment 8 Geert Janssens 2017-04-26 10:05:11 UTC
Looking through the changes added in gnucash 2.6.16, the following commit is the most likely candidate to have started this regression:
https://github.com/Gnucash/gnucash/commit7432ba8c9145bbc4a11fd74437710c9ac82bdf8f

It unconditionally translates the "CURRENCY" namespace. This seems to mess up loading the currencies from file. I haven't figured out yet in what way exactly that is.

On the other hand I can see why only German speaking people are complaining about it: in most languages this string hasn't been translated yet so in the other languages gnucash will continue to use "CURRENCY" untranslated. The only other language that does have a translation for this string is Finnish. So these people could run into the same bug if their system using the Finish locale.

I don't know (yet) why it only happens on Windows though. Nor why I can't reproduce on Windows XP nor on linux.
Comment 9 Geert Janssens 2017-04-26 12:21:55 UTC
Jannick kindly pointed out in private the github link is not working. I see I missed a "/".
So here it is properly (and shortened):
https://github.com/Gnucash/gnucash/commit/7432ba8c9145bbc4a11f
Comment 10 bugzillagnome.10.muellermeier 2017-04-26 19:30:19 UTC
The bug appears also on Arch Linux 4.10.11-1 with german locale (de_DE.UTF-8). 

The workaround of changing the locale to en_US.UTF-8 works also on this platform.
Comment 11 Geert Janssens 2017-05-03 14:24:42 UTC
Another refinement of the issue:
In comment 7 Jannick suggests the bug doesn't happen if he replaces "CURRENCY" in cmdty:space with "ISO4217".

I didn't register at that moment none of my xml data files has CURRENCY in cmdty:space. They all have "ISO4217". As the second string isn't marked for translation it will always work, hence the reason I couldn't reproduce the issue as reported.

However this brings me to the next mystery: so far I haven't figured out yet how I can make gnucash write "CURRENCY" to cmdty:space at all. In all my attempts gnucash always uses "ISO4217"... And reading the code it looks like that's the only way it can be if the data is saved by gnucash. I may be missing something of course.

However before I dig further into this I'd like to ask those who experience this problem: have you previously used external tools to manipulate the gnucash data ?

Aside of this I have a patch ready to fix the problem. It needs some further testing before I can commit it though.
Comment 12 Geert Janssens 2017-05-03 16:11:59 UTC
I have committed a fix to our stable branch. As I haven't been able to reproduce this I would like to ask some of you who are to run a test with tonight's test build. It will be available as of tomorrow from here:
http://code.gnucash.org/builds/win32/maint/
Comment 13 Björn Heide 2017-05-03 17:18:03 UTC
I am on ArchLinux and also see this bug. But I use MariaDB to store my data, not XML-Files.

This is the commodities Table from the DB:

MariaDB [gnucash_XXXXXX]> SELECT * FROM commodities;
+----------------------------------+-----------+----------+----------+-------+----------+------------+--------------+----------+
| guid                             | namespace | mnemonic | fullname | cusip | fraction | quote_flag | quote_source | quote_tz |
+----------------------------------+-----------+----------+----------+-------+----------+------------+--------------+----------+
| a97d6dca0bc5634acde011c90f34cdee | CURRENCY  | EUR      | Euro     | 978   |      100 |          1 | currency     |          |
+----------------------------------+-----------+----------+----------+-------+----------+------------+--------------+----------+

To fix this bug, I issued this SQL statement:

UPDATE `commodities` SET `namespace`='ISO4217' WHERE `guid`='a97d6dca0bc5634acde011c90f34cdee';
Comment 14 hhn10 2017-05-03 17:34:55 UTC
I will try to run the test build on the weekend - I am on sqlite. Is it needed to update the commodities table?
Comment 15 Geert Janssens 2017-05-03 18:02:15 UTC
(In reply to Björn Heide from comment #13)
> I am on ArchLinux and also see this bug. But I use MariaDB to store my data,
> not XML-Files.
> 
> This is the commodities Table from the DB:
> 
> MariaDB [gnucash_XXXXXX]> SELECT * FROM commodities;
> +----------------------------------+-----------+----------+----------+-------
> +----------+------------+--------------+----------+
> | guid                             | namespace | mnemonic | fullname | cusip
> | fraction | quote_flag | quote_source | quote_tz |
> +----------------------------------+-----------+----------+----------+-------
> +----------+------------+--------------+----------+
> | a97d6dca0bc5634acde011c90f34cdee | CURRENCY  | EUR      | Euro     | 978  
> |      100 |          1 | currency     |          |
> +----------------------------------+-----------+----------+----------+-------
> +----------+------------+--------------+----------+
> 
> To fix this bug, I issued this SQL statement:
> 
> UPDATE `commodities` SET `namespace`='ISO4217' WHERE
> `guid`='a97d6dca0bc5634acde011c90f34cdee';

Yes, I should have mentioned the sql backends as well. From my research I found these do store the namespace as CURRENCY instead of ISO4127, so the opposite of the xml backend. This means the issue will affect everybody who's using the SQL backend with gnucash currently in either German or Finnish.

Your workaround will work until you add a new currency. That will automatically use CURRENCY again. However the patch I have applied should make your change no longer needed. In addition your workaround will circumvent the bug and hence also the fix. So if you want to test my fix and report on it, please do so after you reset the namespaces back to CURRENCY.
Comment 16 Geert Janssens 2017-05-03 18:03:15 UTC
(In reply to hhn10 from comment #14)
> I will try to run the test build on the weekend - I am on sqlite. Is it
> needed to update the commodities table?

No. The fix should make gnucash work again with the unaltered commodities table.
Comment 17 Jannick 2017-05-03 18:08:26 UTC
Geert, first of many thanks for all your efforts you are spending on this. Much appreciated!

IIRC the reported issue appeared after I converted an sqlite db to xml format (for performance reasons). That fits well to what you are saying in comment #15.
Comment 18 Heinz Repp 2017-05-04 11:28:35 UTC
I am the reporter on the German mailing list. Downloaded the brand new http://code.gnucash.org/builds/win32/maint/gnucash-2.6.16-2017-05-04-git-9087418+-setup.exe, installed it and tried to run it, but ran into a new error:

gnucash.exe - system error: the program could not be started because icuin57.dll is missing - reinstall to fix the problem.

Needless to say, reinstall didn't solve this. Vanilla 2.6.16 does not have this isssue. Did you introduce a new dependency without providing it? System is Windows 10 pro x64 ver. 1607. Dependency Walker tells me ICUIN57.DLL (and ICUUC57.DLL, also missing) are used by LIBWEBKITGTK-1.0-0.DLL, which is used by LIBGNCMOD-HTML.DLL.

Please fix this dependency issue, can't wait to test your currency fix!
Comment 19 Geert Janssens 2017-05-04 12:36:53 UTC
Thanks for the feedback Heinz. Looks like the work John has been doing to update webkit introduced this new issue. I'll check into it.
Comment 20 Geert Janssens 2017-05-04 14:30:17 UTC
I have pushed a change that should include icu in the next installer. I haven't been able to test this myself yet so let's see what tomorrow's nightly does.
Comment 21 Geert Janssens 2017-05-05 10:22:13 UTC
John started a build earlier yesterday instead of waiting for the nightly build to run. Due to some internals of the build process this new installer has the same name as the one referred to in comment 18 (you may note the installer is slightly bigger than older ones)
http://code.gnucash.org/builds/win32/maint/gnucash-2.6.16-2017-05-04-git-9087418+-setup.exe

So if you download this now it should include the fix.
Comment 22 Jannick 2017-05-05 13:13:47 UTC
Here the nightly built version from yesterday afternoon works like a charm (xml, Win10, German version).

However, here a new thingy when converting between .xml and .sqlite. I am not sure if this is related to our original issue - or is anything missing in the nightly build to convert to .sqlite?

1 - .sqlite > .xml: works fine and cmdty:space is ISO4217 (not very sure if this was different before).

2 - .xml > .sqlite: error prompted with translated message from https://github.com/Gnucash/gnucash/blob/master/po/de.po:11502:

#: ../src/gnome-utils/gnc-file.c:336
#, c-format
msgid ""
"The server at URL %s experienced an error or encountered bad or corrupt data."
msgstr ""
"Der Server unter der URL »%s« stiess auf einen Fehler oder benutzte falsche "
"oder fehlerhafte Daten."
Comment 23 John Ralls 2017-05-05 14:19:43 UTC
Someone reported the SQLite problem on Master nightlies a few days ago. If it also affects maint nightlies that rules out the new sql backend in master and points to a build problem, possibly with SQLite3. I just checked and we haven't changed the SQLite version on the Windows build since we first added the SQL backends 9 years ago, so it isn't that either.
Comment 24 Heinz Repp 2017-05-06 10:09:06 UTC
(In reply to Geert Janssens from comment #21)
> ...
> http://code.gnucash.org/builds/win32/maint/gnucash-2.6.16-2017-05-04-git-
> 9087418+-setup.exe
> 
> So if you download this now it should include the fix.

I confirm that this new installer now runs flawlessly on my Win 10 Pro x64 box and does not exhibit this annoying alert on missing account currency any more.

My vote: change status to wfm.

Heinz
Comment 25 Björn Heide 2017-05-06 11:59:25 UTC
(In reply to Geert Janssens from comment #15)
> 
> Your workaround will work until you add a new currency. That will
> automatically use CURRENCY again. However the patch I have applied should
> make your change no longer needed. In addition your workaround will
> circumvent the bug and hence also the fix. So if you want to test my fix and
> report on it, please do so after you reset the namespaces back to CURRENCY.

Just build gnucash from git branch maint and changed namespace back to CURRENCY and I no longer see any "Kontowährung konnte nicht bestimmt werden." messages.

So far your fix works for me. Thank you.

Björn (on Arch Linux with LANG=de_DE.UTF-8)
Comment 26 Geert Janssens 2017-05-20 15:22:54 UTC
(In reply to Jannick from comment #22)
> However, here a new thingy when converting between .xml and .sqlite. I am
> not sure if this is related to our original issue - or is anything missing
> in the nightly build to convert to .sqlite?
> 
> 1 - .sqlite > .xml: works fine and cmdty:space is ISO4217 (not very sure if
> this was different before).
> 
> 2 - .xml > .sqlite: error prompted with translated message from
> https://github.com/Gnucash/gnucash/blob/master/po/de.po:11502:
> 
> #: ../src/gnome-utils/gnc-file.c:336
> #, c-format
> msgid ""
> "The server at URL %s experienced an error or encountered bad or corrupt
> data."
> msgstr ""
> "Der Server unter der URL »%s« stiess auf einen Fehler oder benutzte falsche
> "
> "oder fehlerhafte Daten."

This is another bug that happened to be introduced in 2.16. I have filed a separate bug report for this one: bug 782868
Comment 27 Geert Janssens 2017-05-20 15:24:05 UTC
Thanks for the feedback all. I'm keeping the bug open for a bit longer as I intend to tweak my initial fix before calling it final. I may need some testers again at that point :)
Comment 28 Joachim Langenbach 2017-06-01 05:53:26 UTC
*** Bug 783285 has been marked as a duplicate of this bug. ***
Comment 29 Heinz Repp 2017-06-02 10:17:40 UTC
The bug is also prevalent in Debian's experimental 2.6.16 build and OpenSuse's (see above) - so "Hardware" is not only Windows, it is "all" - please change (as I seem to be not able to do so ...)!
Comment 30 Jannick 2017-06-05 23:22:03 UTC
Removed reference to Windows from subject.
Comment 31 Geert Janssens 2017-06-20 20:39:34 UTC
I have amended my changes with a few more commits. Can any of you who was affected
by this bug test tomorrow's nightly build or build/test the current maint branch ?
Just to be sure I didn't create any regressions...
Thanks!
Comment 32 Jannick 2017-07-01 11:36:02 UTC
Version https://wiki.gnucash.org/builds/win32/maint/gnucash-2.6.16-2017-06-30-git-b4520d6+-setup.exe works well on my Win10 box.

Sorry for the delay & many thanks, Geert!
Comment 33 Geert Janssens 2017-07-01 12:00:37 UTC
That's ok. Thanks for the feedback!
Comment 34 John Ralls 2017-10-17 21:42:52 UTC
A user, sefnot, reported on IRC that he'd wound up with two entries in his commodity table for Euros:

'44464ab1b91ff1f930f48fe6b81ee760', 'WÄHRUNG', 'EUR', 'Euro', '978', 100, 1, 'currency', ''
'152fea8aea8fbe8290df4d158315e0bb', 'CURRENCY', 'EUR', 'Euro', '978', 100, 1, 'currency', ''
Comment 35 John Ralls 2017-10-17 23:17:37 UTC
As reported on IRC the user created a database with 2.6.15; he may have used multiple locales because he has a price table entry with conversions between the 4446 and 152f currencies from before upgrading to 2.6.16. He found today that after upgrading to 2.6.18 he was unable to post an invoice because his A/R was in one currency and his income account in the other.
Comment 36 Geert Janssens 2017-10-18 08:31:39 UTC
I can imagine how the two currencies got introduced in 2.6.16 and how it would trigger conversion rates between the two. I can't though for 2.6.15. So my best guess is the conversion got added to the price table for a transaction that was either created or altered using gnucash 2.6.16 but dated before upgrading to that version.

It may not really matter though how it got there. The most important thing to take from this report is that we'll need to modify our check & repair code to detect double currencies and reduce this back to one.
Comment 37 Heinz Repp 2017-10-18 12:24:59 UTC
I can only add what I did after the fixed 2.6.16-2017-05-04-git-9087418+ appeared that fixed this bug. My setup uses a MariaDB database, so I could do my maintenance tasks with mysql command line - a GUI mysql client will do also.

The commodities table had two mnemonic "EUR" commodities, one with namespace "CURRENCY" and one with namespace "WÄHRUNG" - the latter one introduced through this bug. I noticed the two guids then replaced all appearances of the "WÄHRUNG" guid by the "CURRENCY" guid within the prices (columns commodity_guid and currency_guid) and transactions tables (column currency_guid) - there were only some records near the end. As I did not create any accounts with the faulty version, I needed no corrections to the accounts table (commodity_guid). Then I deleted the commodities row with namespace "WÄHRUNG" and all prices rows where currency_guid=commodity_guid (they all had value_num=value_denum, meaning 1.00). That was it.

In short:
1. Locate your surplus localized currency row in commodities table
2. Replace all occurrences of its guid with the corresponding "CURRENCY" row's guid within transactions, prices and accounts tables, fields currency_guid and commodity_guid
3. Delete the surplus record in commodities table
4. Delete all identical rows in prices table - this is cosmetic

Hth

Heinz
Comment 38 John Ralls 2018-06-29 23:56:06 UTC
GnuCash bug tracking has moved to a new Bugzilla host. The new URL for this bug is https://bugs.gnucash.org/show_bug.cgi?id=781634. Please continue processing the bug there and please update any external references or bookmarks.