GNOME Bugzilla – Bug 571447
libofx (external) crashes on invalid date format in DTSTART/DTEND
Last modified: 2018-06-29 22:17:58 UTC
Steps to reproduce: 1. On Windows Gnucash 2.2.8, create a new empty account system 2. Do File>Import>OFX of an OFX file containing <BANKTRANLIST><DTSTART>03/11/2008</DTSTART><DTEND>30/11/2008</DTEND> You get an immediate crash Here is the entire OFX file OFXHEADER:100 DATA:OFXSGML VERSION:102 SECURITY:NONE ENCODING:USASCII CHARSET:1252 COMPRESSION:NONE OLDFILEUID:NONE NEWFILEUID:NONE <OFX><SIGNONMSGSRSV1><SONRS><STATUS><CODE>0</CODE><SEVERITY>INFO</SEVERITY></STATUS><DTSERVER></DTSERVER><LANGUAGE>ENG</LANGUAGE><FI><ORG>MyBank</ORG><FID>My Bank</FID></FI></SONRS></SIGNONMSGSRSV1><BANKMSGSRSV1><STMTTRNRS><TRNUID>1</TRNUID><STATUS><CODE>0</CODE><SEVERITY>INFO</SEVERITY></STATUS><STMTRS><CURDEF>EUR</CURDEF><BANKACCTFROM><BANKID>MyBank</BANKID><ACCTID>123-4567890-12</ACCTID><ACCTTYPE>CHECKING</ACCTTYPE></BANKACCTFROM><BANKTRANLIST><DTSTART>03/11/2008</DTSTART><DTEND>30/11/2008</DTEND><STMTTRN><TRNTYPE>DEBIT</TRNTYPE><DTPOSTED>20081128</DTPOSTED><TRNAMT>1234,56</TRNAMT><FITID>0203</FITID><CHECKNUM>0203</CHECKNUM><NAME>John</NAME><MEMO>Some Reason</MEMO></STMTTRN></BANKTRANLIST><LEDGERBAL><BALAMT>+4123,80</BALAMT><DTASOF></DTASOF></LEDGERBAL></STMTRS></STMTTRNRS></BANKMSGSRSV1></OFX> Stack trace: Other information: Note that if you remove the dates from DTSTART and DTEND the crash does not happen.
Reproducible on 2.2.9 using the test file above. Output and stack trace: LibOFX INFO: libofx_proc_file(): File format not specified, autodecting... (Above message occured on Line 4294967295, Column 4294967295) LibOFX INFO: libofx_proc_file(): Detected file format: OFX (Open Financial eXchange (OFX or QFX)) (Above message occured on Line 4294967295, Column 4294967295) LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate SIGNONMSGSRSV1 (Above message occured on Line 1, Column 6) LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate SONRS (Above message occured on Line 1, Column 22) LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate FI (Above message occured on Line 1, Column 130) LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate BANKMSGSRSV1 (Above message occured on Line 2, Column 41) LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate STMTTRNRS (Above message occured on Line 2, Column 55) terminate called after throwing an instance of 'std::out_of_range' what(): basic_string::substr Program received signal SIGABRT, Aborted.
+ Trace 214865
Thread 3060070224 (LWP 29243)
The problem is not a non-empty DTSTART or DTEND tag. The problem is that the values of these tags are in an invalid format. You can find the proper format on http://www.ofx.net/DownloadPage/Downloads.aspx DTSTART/DTEND should hold at date in this format: YYYYMMDDHHMMSS.XXX followed by some optional timezone information This can be shortened to YYYYMMSS But your file contains MM/DD/YYYY as dateformat. Did you get this ofx file from your bank or did you (or some program) create these invalid dates ?
*** Bug 612135 has been marked as a duplicate of this bug. ***
The duplicate bug 612135 more or less confirms this invalid information comes straight from the bank without any manual editing. This is unfortunately not a GnuCash bug, but a libofx bug. It was already reported in libofx' bugtracker since 2006: http://sourceforge.net/tracker/?func=detail&aid=1454059&group_id=61170&atid=496353 Alas, the bug is still there at present. I have added a comment over there that this bug also affects GnuCash users. Hopefully some libofx developer will pick this up.
The files came directly from Dexia Belgium like that, without manual editing. The example was hand-crafted for privacy reasons. A dialog message: 'invalid file format' would have been a lot nicer than the crash. I had grown so used to correcting the files that I didn't even notice that my bank made a change in their files. Their change makes Gnucash ompatible with Dexia Belgium. For me personally this bug is no longer an issue, although other banks may still be affected. For those, the fact that there is a complete crash is imo the main issue and to be solved on the gnucash/libofx side.
Some additional information: this bug happens because the ofx standard requires dates to be formatted as YYYYMMDDHHMMSS.XXX[gmt offset:tz name], where the above example uses DD/MM/YYYY. From the bug in libofx' bugtracker it appears several banks around the world tend to use different date formats. And even worse, as the original example shows, in one file several date formats can be mixed. There's no easy way to have a library universally guess the proper and expected date format in all of these cases. From the libofx bug, it seems to me the libofx maintainers prefer to notify the banks of their wrong use of the ofx standards instead of trying to come up with a complex algorithm that should "magically" parse all these date variants properly. I can't blame them. But I have added a patch to libofx' bugtracker that should prevent libofx from crashing due to an invalid date format. This patch doesn't attempt to parse the date, but will instead simply print an error message and return the reference date (somewhere in 1970). This is probably not perfect, but the best I can do right now. Let's hope this will get integrated in libofx very soon.
My patch got committed to the libofx repository today. The next libofx release will have one opportunity less to crash GnuCash. I don't know when the new release of libofx is to be expected though.
I notice cstim has released libofx 0.9.2 which should contain my patch. Can this bug be closed in the GnuCash tracker now ?
I hope so. I've discovered that I have both libofx CVS write access and also file release priviledges over there, so I can just go ahead and make further releases if we find and fix any bugs in libofx.
Ok, marked as closed. Should the problem still reappear when using the new libofx, feel free to reopen this bug also.
GnuCash bug tracking has moved to a new Bugzilla host. This bug has been copied to https://bugs.gnucash.org/show_bug.cgi?id=571447. Please update any external references or bookmarks.