GNOME Bugzilla – Bug 450276
bus error on OFX import
Last modified: 2018-06-29 21:40:07 UTC
If a broker provides an ofx stream which includes a 0 value dividend income transaction, gnucash svn 16161 crashes with a bus error. Stack trace: Thread 0 Crashed: 0 libgncmod-ofx.dylib 0x02fe9de8 ofx_proc_transaction_cb + 3204 (gnc-ofx-import.c:339) 1 libofx.3.dylib 0x03ff4f60 LibofxContext::transactionCallback(OfxTransactionData) + 240 2 libofx.3.dylib 0x03ffd77c OfxTransactionContainer::gen_event() + 428 3 libofx.3.dylib 0x03ff955c OfxMainContainer::gen_event() + 632 4 libofx.3.dylib 0x04016cc0 OFXApplication::endElement(SGMLApplication::EndElementEvent const&) + 1816 5 libosp.4.dylib 0x0406a814 OpenSP::GenericEventHandler::endElement(OpenSP::EndElementEvent*) + 116 (GenericEventHandler.cxx:156) 6 libosp.4.dylib 0x040bc480 OpenSP::Parser::acceptEndTag(OpenSP::EndElementEvent*) + 784 (parseInstance.cxx:1153) 7 libosp.4.dylib 0x040bf054 OpenSP::Parser::doContent() + 1556 (parseInstance.cxx:162) 8 libosp.4.dylib 0x0407da7c OpenSP::Parser::parseAll(OpenSP::EventHandler&, int volatile const*) + 364 (Parser.cxx:254) 9 libosp.4.dylib 0x0407f180 OpenSP::ParserApp::parseAll(OpenSP::SgmlParser&, OpenSP::EventHandler&, int volatile const*) + 272 (ParserApp.cxx:98) 10 libosp.4.dylib 0x04080314 OpenSP::ParserEventGenerator::run(SGMLApplication&) + 68 (ParserEventGeneratorKit.cxx:194) 11 libofx.3.dylib 0x04008884 ofx_proc_sgml(LibofxContext*, int, char**) + 820 12 libofx.3.dylib 0x03ff823c ofx_proc_file + 4132 13 libofx.3.dylib 0x03ff4988 libofx_proc_file + 848 14 libgncmod-ofx.dylib 0x02fead5c gnc_file_ofx_import + 532 (gnc-ofx-import.c:682) 15 libgncmod-ofx.dylib 0x02feb258 gnc_plugin_ofx_cmd_import + 32 (gnc-plugin-ofx.c:146) 16 libgobject-2.0.0.dylib 0x02b0d37c g_closure_invoke + 400 17 libgobject-2.0.0.dylib 0x02b1e08c signal_emit_unlocked_R + 2676 Steps to reproduce: Start with a clean data file which includes the default investment accounts File/Import/Import OFX>QFX... choose the file ofxcrashJune2007.ofx (attached to bug report) Create commodities for the two stocks when prompted Choose the brokerage account Create stock accounts for the two securities as prompted Choose the dividend income account when prompt Gnucash crashes somewhere in the last 2 steps (I've had it last as long as the income account selection, but sometimes it has failed with the first stock account creation.) If you remove the 0 value dividend income transaction (lines 66-80 in ofxcrashJune2007.ofx), then the crash does not occur. I don't know when ameritrade started including 0 value dividends for a sweep account transaction, but they've certainly screwed up their ofx streams since they changed sweep account stuff around about 6 months ago. Imports did work correctly as of 2.5 months ago.
Created attachment 90506 [details] ofxcrashJune2007.ofx crashes svn 16161 delete lines 66-80, and crash does not occur
Created attachment 90512 [details] [review] Fix the crash Does the attached patch fix the crash? Seems to me the code in gnc-ofx-import.c simply forgot to check whether the security_data_ptr member is non-NULL.
That fixes the import for this file. I get another bus error in the original untrimmed ofx file. Just in case the error is close enough to make sense, I'll include a bit of the crash log here. I will try to find the offending ofx entry and open another bug report. Thread 0 Crashed: 0 libgncmod-generic-import.dylib 0x02f97798 split_find_match + 1080 (import-backend.c:697) 1 libgncmod-generic-import.dylib 0x02f97b4c gnc_import_find_split_matches + 364 (import-backend.c:833) 2 libgncmod-generic-import.dylib 0x02f9836c gnc_import_TransInfo_init_matches + 148 (import-backend.c:1079) 3 libgncmod-generic-import.dylib 0x02f9eb90 gnc_gen_trans_list_add_trans + 208 (import-main-matcher.c:735) 4 libgncmod-ofx.dylib 0x02fe06f8 ofx_proc_transaction_cb + 5556 (gnc-ofx-import.c:520) 5 libofx.3.dylib 0x03ff4f60 LibofxContext::transactionCallback(OfxTransactionData) + 240 6 libofx.3.dylib 0x03ffd77c OfxTransactionContainer::gen_event() + 428 7 libofx.3.dylib 0x03ff955c OfxMainContainer::gen_event() + 632
A possible positive side effect of this patch may be that it fixes 430032 as well. I'll be able to test more completely when I track down the second problem in the full ofx file I have. Testing the patch on the attached file was the first time in a long time that I have been able to clear both check marks in the matcher (neither A nor R checked) and have gnucash actually not put the transaction in the register anyway.
The crash from comment#3 is most likely fixed by r16210. There are a whole bunch of places in (sorry) Benoit's code where it is assumed that pointers are non-NULL, which will immediately crash if that happens to be not the case, e.g. in this import-backend.c:697 crash. I fixed some in r16209 and r16210 (the stacktraces are verbose enough for this), but I hope we can fix the remaining ones soon as well. The transactions will most probably not be imported correctly yet, but at least this won't lead to a crash anymore. Are you sure bug#430032 is indeed fixed as well? I'd happily close that one as duplicate. :-)
I think the transactions causing this round of crashes are ones that I don't want anyway (and maybe they even ignore parts of the ofx spec...). I'll have to do more testing with respect to 430032. I wasn't completely successful with the 16209, and I may be confusing bugs. In any case, I was able to import both my problem files without crashing after r16210 and the libofx patch.
bug#430032 is not a duplicate, but the problem with the specific sample file in that bug is fixed with either r16209 or r16210. See my longer explanation in that bug's comments. There's a more general register refresh-on-import-finish bug that is still present.
what is the current status here?
crash is fixed. Closing this one. Display of unimported transactions still remains.
GnuCash bug tracking has moved to a new Bugzilla host. This bug has been copied to https://bugs.gnucash.org/show_bug.cgi?id=450276. Please update any external references or bookmarks.