GNOME Bugzilla – Bug 711580
QFX Import broke accented characters like "é"
Last modified: 2018-06-29 23:21:06 UTC
Created attachment 259152 [details] Example of file to reproduce the bug Import of .qfx file in gnucash 2.5.7 and in 2.4.13 broke accented characters like "é", but it work perfectly in gnucash 2.4.11. I put in attachment an example of QFX file with french word including accented characters. Ex: "Dépôt" is show as "Dp$t" "Intérêts versés" become "Intrts verss"
Imports and displays correctly using Linux Fedora18. This could be due to whatever font Windows is using. Have you tried changing the Windows fonts? Severity, major? really? Major = Major loss of function in an important area.
I have tried on Windows 7 and Windows XP using default Windows font. No I have not tried changing default Windows fonts. In fact, I never done that, I have no idea how I can do that and it's completely irrelevant since it's the Windows default font. Yes it's major for my point of view. I will continue using 2.4.11 until it's fix since it is completely unreadable. Thank you
"Major loss of function in an important area" Loss of the function "reading the transaction description" in the important area "ledger" definitely qualifies. The reporter is french speaking, at least he will have some readable letters to infer the m,eaning, imagine if he used a completely different alphabet (japanese, chinese, arabic, etc.). One couldn't read anything. As for the cause, most likely it's a mismatch with the libofx version shipped with the windows version, or a regression specific to windows, not a font problem. Can someone try ofxdump on windows with 2.5.3 with the attached file, as well as the output of ofxdump --version?
I still have the same problem but with the new gnucash (2.6.0)
Michel, can you please run the above test?
ofxdump -V => ofxdump 0.9.8 ofxdump test.QFX => <OFX><SIGNONMSGSRSV1><SONRS><STATUS><CODE>0<SEVERITY>INFO<MESSAGE>Authentication Successful.</STATUS><DTSERVER>20131103112618.803[-5:EST]<LANGUAGE>ENG<FI><ORG>INGDIRECTCanada<FID>10951</FI></SONRS></SIGNONMSGSRSV1><BANKMSGSRSV1><STMTTRNRS><TRNUID>0<STATUS><CODE>0<SEVERITY>INFO</STATUS><STMTRS><CURDEF>CAD<BANKACCTFROM><BANKID>0614<ACCTID>3019100772<ACCTTYPE>SAVINGS</BANKACCTFROM><BANKTRANLIST><DTSTART>20130910200000.000[-4:EDT]<DTEND>20131103062618.944[-5:EST]<STMTTRN><TRNTYPE>INT<DTPOSTED>20130930120000.000<TRNAMT>4.9<FITID>397<NAME>Intérêts versés</STMTTRN><STMTTRN><TRNTYPE>CREDIT<DTPOSTED>20131007120000.000<TRNAMT>239.92<FITID>398<NAME>Dépôt<MEMO>WEB BATCH</STMTTRN><STMTTRN><TRNTYPE>CREDIT<DTPOSTED>20131021120000.000<TRNAMT>239.92<FITID>399<NAME>Dépôt<MEMO>WEB BATCH</STMTTRN><STMTTRN><TRNTYPE>DEBIT<DTPOSTED>20131023120000.000<TRNAMT>-382.05<FITID>400<NAME>Retrait<MEMO>Transfert</STMTTRN><STMTTRN><TRNTYPE>INT<DTPOSTED>20131031120000.000<TRNAMT>5.2<FITID>402<NAME>Intérêts versés</STMTTRN></BANKTRANLIST><LEDGERBAL><BALAMT>3333.33<DTASOF>20131103112618.803[-5:EST]</LEDGERBAL><AVAILBAL><BALAMT>3333.33<DTASOF>20131103112618.803[-5:EST]</AVAILBAL></STMTRS></STMTTRNRS></BANKMSGSRSV1></OFX> ofx_proc_status(): Ofx entity this status is relevent to: SONRS Severity: INFO Code: 0, name: Success Description: The server successfully processed the request. Server Message: Authentication Successful. ofx_proc_status(): Ofx entity this status is relevent to: STMTTRNRS Severity: INFO Code: 0, name: Success Description: The server successfully processed the request. ofx_proc_account(): Account ID: 0614 3019100772 Account name: Bank account 3019100772 Account type: SAVINGS Currency: CAD Bank ID: 0614 Account #: 3019100772 ofx_proc_statement(): Currency: CAD Account ID: 0614 3019100772 Start date of this statement: 09/10/13 20:00:00 Est (heure d’été) End date of this statement: 11/03/13 06:26:18 Est Ledger balance: 3333.33 Available balance: 3333.33 ofx_proc_transaction(): Account ID : 0614 3019100772 Transaction type: INT: Interest earned or paid (Note: Depends on signage of amount) Date posted: 09/30/13 12:00:00 Est (heure d’été) Total money amount: 4.90 # of units: -4.90 Unit price: 1.00 Financial institution's ID for this transaction: 397 Name of payee or transaction description: Int®r¬ts vers®s ofx_proc_transaction(): Account ID : 0614 3019100772 Transaction type: CREDIT: Generic credit Date posted: 10/07/13 12:00:00 Est (heure d’été) Total money amount: 239.92 # of units: -239.92 Unit price: 1.00 Financial institution's ID for this transaction: 398 Name of payee or transaction description: D®p$t Extra transaction information (memo): WEB BATCH ofx_proc_transaction(): Account ID : 0614 3019100772 Transaction type: CREDIT: Generic credit Date posted: 10/21/13 12:00:00 Est (heure d’été) Total money amount: 239.92 # of units: -239.92 Unit price: 1.00 Financial institution's ID for this transaction: 399 Name of payee or transaction description: D®p$t Extra transaction information (memo): WEB BATCH ofx_proc_transaction(): Account ID : 0614 3019100772 Transaction type: DEBIT: Generic debit Date posted: 10/23/13 12:00:00 Est (heure d’été) Total money amount: -382.05 # of units: 382.05 Unit price: 1.00 Financial institution's ID for this transaction: 400 Name of payee or transaction description: Retrait Extra transaction information (memo): Transfert ofx_proc_transaction(): Account ID : 0614 3019100772 Transaction type: INT: Interest earned or paid (Note: Depends on signage of amount) Date posted: 10/31/13 12:00:00 Est (heure d’été) Total money amount: 5.20 # of units: -5.20 Unit price: 1.00 Financial institution's ID for this transaction: 402 Name of payee or transaction description: Int®r¬ts vers®s
I run it with ofxdump found in C:\Program Files (x86)\gnucash\bin installed by gnucash 2.6.0
Darn, on linux libofx 0.9.8 parses it correctly. So there really is a charset conversion problem on windows. Can someone familiar with the windows build process check if windows build builds libofx with iconv support?
You can check the libofx-0.9.9 build output in http://code.gnucash.org/builds/win32/build-logs/build-trunk-2014-01-08.log It says: > checking iconv.h usability... yes > checking iconv.h presence... yes > checking for iconv.h... yes > checking for iconv... no > checking for iconv in -liconv... yes However, I don't know what this actually means for iconv support. Also, the libiconv we're using is the one that's being used in the gnome windows binaries, "$GNOME_WIN32_DEPS_URL/libiconv-1.9.1.bin.woe32.zip" That should be a working one, shouldn't it?
It should (iconv is a really mature piece of software). So, either: 1- The terminal test is unreliable (quite possible if UTF-8 isn't the default encoding in the terminal on windows, I remember a time where this was a nightmare on Linux, and the terminal is much more important on Linux...) and libofx is doing what it's designed to do (unconditionally converts strings to UTF-8). thus the problem would be somewhere else. 2- Something specific to windows happens that messes up encoding. OpenSP hasn't been updated in ages, maybe what it sends has problems. Or libofx is doing something wrong on windows. Or gnucash is doing something wrong on windows. I really don't know.
I downgraded to 2.4.11 so the OFX import would work for Japanese UTF-8 encoded files.
Is this bug the same as bug 627773 and bug 639647 ? https://bugzilla.gnome.org/show_bug.cgi?id=627773 https://bugzilla.gnome.org/show_bug.cgi?id=639647
It's not a duplicate of these two bugs, but it is a duplicate of bug 703527 in that on windows the libofx character set conversion is not working properly. It still has to be determined whether this is a problem with how libofx gets built on Windows or really an issue in the libofx source code. Thanks for taking the time to report this. This particular bug has already been reported into our bug tracking system, but please feel free to report any further bugs you find. *** This bug has been marked as a duplicate of bug 703527 ***
GnuCash bug tracking has moved to a new Bugzilla host. This bug has been copied to https://bugs.gnucash.org/show_bug.cgi?id=711580. Please update any external references or bookmarks.