After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 771451 - Chase Visa QFX import null description and memo
Chase Visa QFX import null description and memo
Status: RESOLVED NOTGNOME
Product: GnuCash
Classification: Other
Component: Import - OFX
git-maint
Other All
: Normal major
: ---
Assigned To: gnucash-import-maint
gnucash-import-maint
: 771746 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2016-09-14 21:34 UTC by Lee
Modified: 2018-06-29 23:51 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
chase transaction import null description and memo (37.42 KB, image/png)
2016-09-14 21:35 UTC, Lee
  Details
sample QFX file with modified account number (1.48 KB, text/plain)
2016-09-19 18:06 UTC, Lee
  Details
Patch to the OFX-160 DTD to allow a CATEGORY element. (1.22 KB, patch)
2016-10-01 07:14 UTC, John Ralls
none Details | Review
Patch to LibOFX DTD to allow bogus CATEGORY element. (900 bytes, patch)
2016-10-28 14:00 UTC, John Ralls
committed Details | Review

Description Lee 2016-09-14 21:34:15 UTC
This was working until Chase updated their website earlier.  Now when I download account activity as a QFX and import it the descriptions and memo are null
Comment 1 Lee 2016-09-14 21:35:25 UTC
Created attachment 335573 [details]
chase transaction import null description and memo
Comment 2 John Ralls 2016-09-15 01:04:00 UTC
Please attach the actual QFX file. You may want to edit it first to remove any private information (Account numbers would be the most likely).
Comment 3 Martin F 2016-09-18 21:14:13 UTC
I have the same problem. Here is a sample transaction before and after Chase change:

Before:
<STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20160715120000[0:GMT]
<TRNAMT>-1.50
<FITID>...
<NAME>SOME STORE A
</STMTTRN>

After:
<STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20160808120000[0:GMT]
<TRNAMT>-7.10
<FITID>...
<NAME>SOME STORE B
<CATEGORY>
<MEMO>null
</STMTTRN>

There seems to be two tags added: <CATEGORY> which is empty, and <MEMO>null
Comment 4 Lee 2016-09-19 18:06:35 UTC
Created attachment 335874 [details]
sample QFX file with modified account number

Thanks to previous comment.  I found if you run "grep -v CATEGORY" on the Chase ofx files you get the descriptions back.
Comment 5 John Ralls 2016-09-21 06:17:20 UTC
*** Bug 771746 has been marked as a duplicate of this bug. ***
Comment 6 John Ralls 2016-09-21 06:19:42 UTC
Bug 771745 has attached a full import file with the extra fields for testing.
Comment 7 John Ralls 2016-09-25 15:11:31 UTC
Err, bug 771746 is what has an import file.

I've tested and determined two things: First, the data passed to GnuCash has already had the Name element contents deleted and second that simply removing just the <CATEGORY> element is sufficient. One still gets "null" in the memo but that's comparatively harmless. Moving the <CATEGORY> element to after the <MEMO> element causes the <MEMO> value to be lost rather than the <NAME> element.

I see two feasible hacks: Alter ofx160.dtd so that CATEGORY is legal or add a special case to libofx's sanitize_proprietary_elements() so that it's removed. While we could pretty easily patch the Windows and Mac builds of libofx to do either one it's not a solution for Linux users.
Comment 8 John Ralls 2016-10-01 07:14:17 UTC
Created attachment 336718 [details] [review]
Patch to the OFX-160 DTD to allow a CATEGORY element.

Patching the dtd works without needing to recompile so it's a feasible work-around for everyone. Find share/libofx/dtd/ofx160.dtd. Most Linux users will find it in /usr, Mac users will find it in Gnucash.app/Contents/Resources/share (Macports: /opt/usr, Fink: ~/sw, Homebrew: /usr/local), Windows users in C:\Program Files (x86)\gnucash\. Change to that directory.

Download the attached patch and apply it with
  patch -p0 < ofx-dtd-chase.patch

Windows users who don't have Video Studio or Cygwin/MSys installed will need to edit the file directly. Use Notepad. Find the lines with a '-' in the left margin of the patch and replace them with the corresponding lines with a '+' in the left margin.
Comment 9 Martin F 2016-10-08 15:56:17 UTC
I can confirm that the patch in comment 8 is working.

It seems Chase also fixed their bug and no longer has CATEGORY element any more (last checked today.) Their MEMO category also now has valid entry instead of 'null'.

You can also just re-download the qfx file and it should have fixed format by now.

Maybe this should be marked as not a bug of gnucash? I verified OFX 1.6 version spec and it does not mention CATEGORY element, so this would only be just a workaround to Chase bug (no pun intended.)
Comment 10 Philip Walden 2016-10-08 19:22:23 UTC
Not seeing the correction from Chase on my two credit cards (yet).
Comment 11 Martin F 2016-10-16 14:58:00 UTC
My bad. You are right that Chase still has the bug with credit cards. So the workaround is still required.
Comment 12 J. David Beutel 2016-10-28 08:06:49 UTC
I agree, the patch in comment 8 works.  I had to apply it manually, with GnuCash 2.4.7, because it failed to apply when I tried 'patch < ofx-dtd-chase.patch'.  Also, with -p0, the absolute path didn't match at all.

The memo is "null", but that's what Chase is providing in its download.
Comment 13 John Ralls 2016-10-28 14:00:41 UTC
Created attachment 338724 [details] [review]
Patch to LibOFX DTD to allow bogus CATEGORY element.

I've edited the patch so that it should apply correctly with `patch -p1 < libofx-dtd-chase.patch`. You'll need to navigate to the installation prefix first:
Microsoft Windows: C:\Program Files (x86)\gnucash
Macs with GnuCash.app: Gnucash.app/Contents/Resources
Macs with MacPorts: /opt/local
Linux when installed with a package manager: /usr
Comment 14 Geert Janssens 2018-04-30 18:44:17 UTC
I have submitted a fix upstream to libofx for this particular case of an invalid CATEGORY tag. The fix got applied there today. Hopefully a new libofx release will follow soon...
Comment 15 Geert Janssens 2018-06-21 13:47:01 UTC
Libofx has released version 0.9.13 incorporating this fix (and a few others).

Time to close this bug as NotGnuCash (being NotGnome in this tracker).
Comment 16 Geert Janssens 2018-06-21 13:47:52 UTC
Comment on attachment 338724 [details] [review]
Patch to LibOFX DTD to allow bogus CATEGORY element.

Mark as committed though this commit happened in libfox rather than gnucash.
Comment 17 John Ralls 2018-06-29 23:51:07 UTC
GnuCash bug tracking has moved to a new Bugzilla host. This bug has been copied to https://bugs.gnucash.org/show_bug.cgi?id=771451. Please update any external references or bookmarks.