GNOME Bugzilla – Bug 114724
QIF importer drops split details for transfers with identical amount
Last modified: 2018-06-29 20:33:23 UTC
When I have a transaction that has a multipart split, and the split involves transfers to two different account, each for the same $ amount, gnucash keeps only 1 on import. If the $ amount differs, both will be taken.
Created attachment 17321 [details] sample qif file demonstrating problem (from Mac Quicken 2003)
It looks like this is the transaction in question, correct? D6/8/03 PPay M T100.00 A A A A A A LSalary SSalary E $300.00 S[401k] E $-100.00 S[ESPP] E $-100.00 ^ If I just load this transaction on its own then the parser loads it just fine and it gets Split properly, so there clearly isn't anything wrong with the parser. But there is certainly something else going on. What is the "source account" of this data file?
Created attachment 17334 [details] the gnucash file obtained after loading the previously attached qif file
I observed this behaviour when loading my actual qif file into gnucash. To generate a simpler example, I had quicken create a new file, to which I added three accounts -- checking, a bank account; 401k, a portfolio account, and ESPP, a cash account. I created a single transaction, a mock pay cheque for $300, of which $100 went to the 401k account and $100 to the ESPP account, leaving $100 in the checking account. When loaded into gnucash, I get $200 in checking, $100 in 401k and $0 in ESPP. Though this example doesn't demonstate it, based on my full qif file, this behaviour only happened when the transfer amounts were identical; when they differed, the split was entered as expected.
Very strange. The fact that different amounts are treated differently is just..... STRANGE!
I have gone back to the original quicken file and added a second transaction, again for $300 in salary, $100 to the 401k but only $50 to the ESPP account. All parts are added, though the ESPP shows up under a split transaction in the checking account, while the 401k part as a separate line item, which is a bit strange...
Created attachment 17373 [details] qif file with a second transaction, differing transfer amounts
Created attachment 17374 [details] resulting gnucash xml file, after loading 2-transaction qif file
I made another discovery. If I rename the "401k" account "Retirement" or "My 401k", the split is handled correctly, but not if I rename it "401k account". So, something to do with the name starting with digits? It's a bit odd because it's the ESPP split line that gets dropped. And no matter the name, the transfer to that account does not show up as part of the split it seems, but as a separate line in the checking register -- maybe it has something to do with the account being a portfolio?
I'll take ownership, although I can't guarantee anything being fixed.
I tried changing all the names of my accounts in my original quicken file, so that none started with a digit, and there were no non alphnumeric characters. Unfortunately, the problem remains. In practice, a pay cheque would have on the order of a dozen or so split entries, and in the cases where the 401k deduction matched the espp deduction, the espp deducution was lost. The previous suggestion that an account name that starts with digits is related is probably a red herring, though the specifics of the account name may still be involved somehow.
Having explored, the problem appears to be the order in which the transactions are listed in the file. By changing the name of the account you probably changed the order in the file such that the split transaction occurred earlier in the file.. I think the problem is the import not properly merging transactions.
fixed summary
Another thing I noticed... If you change your 401k account from an Investment to a Bank account type it seems to import correctly... I'll also note that the problem here is not that the 401k split has the same value as the ESPP split -- if I change both the 401k split and the '401k account transaction' to be 100.01 instead of 100.00, it still imports incorrectly. So the problem is not matching the ESPP value.
Does this issue also occur in the 2.0.x versions? Development on 1.8.x has stopped, so please upgrade to 2.0.x (most current is 2.0.1) and see whether this problem still occurs.
I suspect this bug still exists in 2.0, but I haven't tested it.
Any news on this one?
Created attachment 99947 [details] Additional QIF file to demonstrate bug as seen in 2.2.1
The additional QIF file that I have attached demonstrates the bug in 2.2.1: 1. From a savings account, withdraw $300 cash and pay a $1.50 ATM fee (a split transaction) 2. From a checking account, on the same day, withdraw $300 cash 3. The split transaction is imported perfectly. 4. The bug: the second transaction, a withdrawal from checking, is instead imported as a second withdrawal from savings. NOTE: if the order of the last two transactions in the file are reversed, the import works correctly.
Please ignore my two previous comments, because the example I gave is now known to be a separate bug. Filed as bug #506810.
Created attachment 101975 [details] Additional QIF file to demonstrate bug as seen in 2.2.2 This test file was provided to the mailing list by William Hamblen.
Created attachment 102026 [details] [review] Proposed patch for testing & comment The patch is that the merging code that was at the bottom has been moved to the top, allowing all split transactions to have priority over those with actions and securities.
I want to try to describe how the QIF importer is currently working, how the bug occurs, and what the patch does... In QIF files, transfers between accounts show up as two separate entries: one in the "from" account, and one in the "to" account. The QIF importer has to look for these two "splits" and match them together to avoid creating duplicate transactions in GnuCash. Once a match is found, the QIF importer must decide which split to keep and which to discard. The way it currently operates, if a transfer involves an investment account then the split in the investment account always gets priority. The other split of the matching pair gets discarded. If that other half happens to be part of a QIF split transaction (one with "S" lines), then that means that the QIF importer is discarding one of those split lines, thereby mangling the QIF split transaction. This is the bug. What the patch I wrote does is to change the rules, so that if a QIF split transaction is involved in a match then it always has priority. The other half of the match will always be the half that gets discarded, even if it is from an investment account. I tested this patch on the most recent QIF file that I attached (from William Hamblen) and it fixed the problem. I also tested this patch on 15 years of my own personal data, and it didn't seem to break anything else. However, seeing as I have changed what would seem to be a rather sensitive spot in the code, I'd appreciate any/all feedback, test results, or possible unintended consequences. Thanks!
I've tested this on my real data (17 years worth of Quicken data exported from Quicken Basic 2004 to one gigantic QIF) and everything looks perfect. I've checked share and dollar balances for all non-zero Quicken accounts (but not categories) and the imported GnuCash accounts match in every case. At least for my data, this patch is a tremendous improvement. Thanks Charles!
Hi, for the future, it helps if you run a diff from the top-level of the tree so we don't need to figure out where in the tree the patch needs to apply. I've applied this to trunk as r16828. Awaiting backport. Thanks! Note that I still have a concern that if there's an investment account transaction involved you might lose the investment information, like shares/price.
branches/2.2 @ r16830 for GnuCash 2.2.3. Thanks a lot!
Derek, when I wrote the patch I was having the same concern, i.e. can investment information be lost? But looking at all the special cases that were set out, I still couldn't think of an example where a QIF split transaction that includes a QIF account transfer would involve a security. It worked for two of us, but if it doesn't work for others then maybe some code to copy the investment stuff could be tacked on. I guess we'll have to wait and see. About the diff, I assume that by the top of the tree you mean one level above "src"?
Yeah, I mean, effectively, "svn diff" from the top level (trunk, or branches/2.2) directory.
GnuCash bug tracking has moved to a new Bugzilla host. This bug has been copied to https://bugs.gnucash.org/show_bug.cgi?id=114724. Please update any external references or bookmarks.