After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 754856 - scheduled transaction <gnc.app-utils.sx> fails without warning
scheduled transaction <gnc.app-utils.sx> fails without warning
Status: RESOLVED FIXED
Product: GnuCash
Classification: Other
Component: Scheduled Transactions
2.6.7
Other Windows
: Normal major
: ---
Assigned To: gnucash-core-maint
gnucash-core-maint
Depends on:
Blocks:
 
 
Reported: 2015-09-11 00:18 UTC by David Carlson
Modified: 2018-06-29 23:43 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Trace file for an instance of failure (453 bytes, text/plain)
2015-09-16 14:17 UTC, David Carlson
Details
A trace file from release 2.6.9 with same problem (659 bytes, text/plain)
2015-10-16 16:46 UTC, David Carlson
Details
the transaction that fails (85.89 KB, image/png)
2015-11-14 17:14 UTC, David Carlson
Details
the slr entry for that transaction (39.48 KB, image/png)
2015-11-14 17:15 UTC, David Carlson
Details
trace from 2.6.10 nightly build Jan 2 2016 (307 bytes, text/plain)
2016-01-02 19:19 UTC, David Carlson
Details
Bad SX before SLR (79.66 KB, image/png)
2016-01-26 20:13 UTC, David Carlson
Details
SLR Dialog (35.97 KB, image/png)
2016-01-26 20:14 UTC, David Carlson
Details
Created transaction with extra amount (86.08 KB, image/png)
2016-01-26 20:15 UTC, David Carlson
Details

Description David Carlson 2015-09-11 00:18:46 UTC
Prior to release 2.6.7 I was able to save and later edit scheduled transactions that contained several split lines where each split line had a value of zero and the Create Automatically box was checked on the Overview tab.

Starting with release 2.6.7 I cannot edit the notes or memo fields of that type of scheduled transaction and then click OK.  I get an error message "Scheduled Transactions with variables cannot be automatically created.", possibly because I have selected Create automatically on the Overview tab, but I cannot see that from the Template Transaction tab where I am editing and trying to save the revised text.

Strangely, if I cancel out and return later, I find that the text actually was changed, which should not have happened since I cancelled out.  That was what I really wanted, so I am glad that it happened that way this time, but if it refuses to accept a change, the change should not happen either.

If it was possible to temporarily uncheck Create Automatically to save the textual change then later check Create Automatically and save successfully without triggering the error, that could be a work-around, but I won't try that in my working file.
Comment 1 David Carlson 2015-09-11 00:23:52 UTC
I am not sure how the internal logic is able to detect that a variable is present when there are no alpha characters in any of the value boxes.  It must have a strange algorithm to do that.
Comment 2 David Carlson 2015-09-15 18:56:20 UTC
I have another scheduled transaction that does not contain any variables but it is scheduled to be created automatically a few days ahead and also to be reminded a few more days ahead, adding up to November 1st as of today September 15.  In this case, every split line has a numeric value, one line has a simple equation one value minus another value minus a third value.  A reminder appeared in Since Last Run.  It wanted a price for (null) => USD.  I entered zero.  I always check the box to show transactions.  The transaction did not appear in the list of created transactions nor did it appear in the bank account register.  However, the Scheduled transaction edit dialog did show that it had been created for November 1 2015 and was now scheduled for December 1.

I closed the file without saving and repeated the SLR, this time entering a value of 1, which is what I usually do when I think it should not even be asking.  This time the transaction again did not appear in the bank register, but the Scheduled transaction edit dialog again showed that it had been created.

It has worked fine in release 2.6.6 and earlier for about 18 months without any need for any currency price.

I suspect that this is related even though it is a very different symptom.
Comment 3 David Carlson 2015-09-15 19:28:33 UTC
I failed to mention that it was also necessary to change the reminder status to TO do as expected to trigger the entry, or lack of expected entry in this case.
Comment 4 David Carlson 2015-09-16 14:17:21 UTC
Created attachment 311460 [details]
Trace file for an instance of failure

The tracefile shows the failure.
Comment 5 David Carlson 2015-09-16 14:24:43 UTC
When I attached the tracefile the rest of my comment disappeared.

The scheduled transaction that caused the failure contains 5 split lines.  One is a bank account, one is an income account and three are expense accounts, all are denominated in USD.  As I mentioned above, there are no variables, there is one simple equation of subtraction of numbers.  Finally, I have no idea where Null Account came from.
Comment 6 David Carlson 2015-10-16 16:43:38 UTC
I have another tracefile since the same transaction again silently failed to appear this month.  Again the transaction is marked as having been entered, but it is not in the register.  It still has a mysterious line in the SLR dialog asking for a null value.  This is now occurring in release 2.6.9.
Comment 7 David Carlson 2015-10-16 16:46:03 UTC
Created attachment 313488 [details]
A trace file from release 2.6.9 with same problem

I have another tracefile since the same transaction again silently failed to appear this month.  Again the transaction is marked as having been entered, but it is not in the register.  It still has a mysterious line in the SLR dialog asking for a null value.  This is now occurring in release 2.6.9.
Comment 8 John Ralls 2015-10-30 19:58:38 UTC
It looks like you've jammed a bunch of different complaints into a single bug report:

1) Multi-commodity SXes raise a warning about not being able to auto-create SXes with variables. The condition in the code actually raises that warning if there's a split in the SX with a different commodity from the default (which is set from the first split, just like with regular transactions).
2) The SLR dialog wants a price even if the amount on the split is 0.
3) The auto-create warning doesn't prevent saving the SX.
4) That a different SX that worked OK in 2.6.6 no longer works.

I can make an exception to the multi-commodity rule for 0-amount splits, and do the same for the SLR dialog's exchange rate logic if I can figure out the latter, and I can make the warning say "variables or more than one commodity". That particular code is quite old, so it's hard to believe that it worked any differently in 2.6.7

The save behavior is probably because the edits work directly on KVP entries and aren't affected by the actual save. That's not a defense, it's definitely bad design.

The errors in the two attachments indicate that you have a defective SX, probably a split with an empty or invalid Account. That would also explain the request for a (null)=>USD exchange rate. If that's not the case, please attach a screenshot of that SX in the editor.
Comment 9 John Ralls 2015-10-30 20:02:45 UTC
I think that problem 2 is the same as both Bug 764192 and Bug 662126, which you also submitted. Do you agree?
Comment 10 David Carlson 2015-11-14 17:14:41 UTC
Created attachment 315605 [details]
the transaction that fails

Yes, problem 2 is covered elsewhere, and I think 3 is the one about the auto creation not working with variables error warning happening when the SLR is run instead of when the SX is saved.

However, I have one transaction that has been entered by the SLR correctly every month for about 18 months until release 2.6.7 came along and now that transaction wants a value for (null) in release 2.6.7 and also release 2.6.9 for windows when it comes due in the SLR.

Per your request I am attaching an image in PNG format, if it works.
Comment 11 David Carlson 2015-11-14 17:15:37 UTC
Created attachment 315606 [details]
the slr entry for that transaction
Comment 12 David Carlson 2015-11-14 17:50:54 UTC
Oops, I just noticed the extra line that must have added itself a couple of months ago, maybe when a certain red Doberman nudged my hand while I was using the computer.

So now I am requesting (1)that the SX editor should have rejected that transaction instead of accepting it in the first place.
(2) when the SLR encounters that error, it should not update the last occurrence date and (3) the SLR should give a user warning when it does not enter the transaction into the register.
Comment 13 John Ralls 2015-11-14 18:40:29 UTC
More like 5, 6, and 7, right?
Comment 14 David Carlson 2015-11-15 21:30:24 UTC
I suppose each item would require different code to implement, but they are very closely related and it would seem silly to me to implement them separately unless there was some infrastructure issue making that very difficult.
Comment 15 John Ralls 2015-11-15 21:41:03 UTC
You missed the point. In comment 8 I enumerated 4 separate issues that you originally raised in this bug. In comment 13 I'm suggesting that you're raising three new issues. Do you agree or do you think that your requests in comment 12 map to the original four?
Comment 16 David Carlson 2015-12-15 21:31:54 UTC
Ok.  I thought before that most of the first three or four issues were already addressed in other bug reports.   I thought that what you call 5, 6 and 7 were suggestions for possible docile behavior in case the underlying problems could not be fixed easily.

In any case, another month has gone by and the bad transaction came up again and failed again, this time without the extra line that you see in https://bugzilla.gnome.org/attachment.cgi?id=315605.

So there is still something wrong with that transaction too.  

Sometimes I aggregate bugs into a single report if i think that one excursion into the code would efficiently address all the issues.  When I am wrong about that, it does make sense to split them into separate reports.
Comment 17 John Ralls 2015-12-30 20:07:42 UTC
Can you test it again with 2.6.10 and a copy of your backup from before the problem SX ran?
Comment 18 David Carlson 2016-01-02 19:19:43 UTC
Created attachment 318170 [details]
trace from 2.6.10 nightly build Jan 2 2016

This trace is from a test of "This copy was built from git rev c2598f8+ on 2016-01-02" nightly build opening a backup that had the suspect transaction ready to create in the SLR.  Running the SLR, the transaction again failed to be created.
Comment 19 John Ralls 2016-01-02 19:27:27 UTC
Hmm. You're sure the red dobe's split is really removed? Is it in the template transaction in the XML?
Comment 20 David Carlson 2016-01-02 19:36:45 UTC
Yes.  I anticipated that my rambunctious red Doberman Retriever would try to jump on my lap again so I reviewed the transaction with the scheduled transaction  editor and the spurious line is not there.
Comment 21 David Carlson 2016-01-02 19:43:47 UTC
In accordance with testing  for https://bugzilla.gnome.org/show_bug.cgi?id=754192 the attachment for comment 11 no longer appears.
Comment 22 John Ralls 2016-01-02 19:54:32 UTC
So it doesn't ask for a price, which is good, but it also doesn't get created, which is not so good. It seems that the split is still in the XML template transaction and doesn't show in the SX editor. (You're *really* sure that it doesn't show as a completely blank line?)

So I guess comment 12 gets another item, make sure that completely empty splits are removed, not just hidden.
Comment 23 David Carlson 2016-01-02 19:58:11 UTC
Oh.  There is a line with an "n" in the reconciled box, but nothing else.
Comment 24 John Ralls 2016-01-02 20:05:29 UTC
Ah. Nuke that with the delete button and try again.
Comment 25 David Carlson 2016-01-02 20:16:25 UTC
Yes!  Going back to a fresh copy of the backup and re-saving the transaction in the SX editor with the highlight on one of the desired lines eliminated that extra line.  Then the transaction was created by the SLR!  Thank you for your patience with me.
Comment 26 John Ralls 2016-01-02 20:22:56 UTC
Excellent.

I'll leave the bug open to remind me to fix the editor to reject the SX if there's a split with no account and to not save anything until all of the errors are cleared.
Comment 27 David Carlson 2016-01-20 05:30:00 UTC
I  am not sure how, my datafile has not been well controlled recently, but somehow the extra line returned to that transaction and release 2.6.11 still does not import it or give an error.  I guess that is why you left the bug open.  If the extra line returns again without my deliberately creating it, there may be some other bug.
Comment 28 John Ralls 2016-01-20 05:40:45 UTC
Yup. I was trying to get the SX editor dialog to reject a transaction with a no-account split and couldn't figure out where the split account gets evaluated. I needed to get the QIF fix out, the whining was deafening, so this bug didn't make that release. Then Real Life intervened and I haven't gotten back to it.

But if you can't keep the bad split from reinserting itself I guess I should work on it from the other end, i.e. the SLR dialog's processing of the split, first.
Comment 29 David Carlson 2016-01-20 17:04:33 UTC
It is possible that I forgot to fix the transaction in my data file after fixing it in the test file, so it is too early to know whether it really reinserts itself.  I have been migrating data files to a different server and some things are getting confused.  While fixing the transaction I noticed that it was terribly easy to accidentally recreate the extra line, so I agree with fixing all areas.
Comment 30 John Ralls 2016-01-22 23:23:50 UTC
Comment on attachment 315606 [details]
the slr entry for that transaction

Well, it turns out that it's not really without warning, it's just that the warning is in gnucash.trace. It says "CRIT <gnc.app-utils> Null account kvp value for SX [AbbVie, Inc Pension Mthly -SBOL 4215], cancelling creation." Go have a look.

It would be better if that error was raised in a dialog, but that's hard, at least for now. There's no signalling in place between that module and the GUI to set the dialog.
Comment 31 John Ralls 2016-01-22 23:41:58 UTC
Nope, I'm wrong, there is an error-signalling mechanism, it's just not used. It's also not good enough to prevent incrementing the SX instance, but that part's fixable.
Comment 32 John Ralls 2016-01-24 22:51:23 UTC
I've implemented warning dialogs when you edit and when the SLR dialog runs the SX so the "without warning" part is completely gone. Unfortunately the design is such that the edits have already been put in memory by the time you click OK in the editor, so even clicking cancel won't undo them. You either have to fix it and click OK again or quit without saving. Nothing I can do about that until we have a real DB backend that we can use for rollbacks.

The warnings will be in tomorrow's nightly for you to try out. I still have to figure out how to keep the SLR from incrementing the instance.
Comment 33 John Ralls 2016-01-25 00:20:31 UTC
The last step wasn't that hard, so give it a try with tomorrow's nightly. Note that because of the lack of a proper rollback and the fact that SX edits are saved to the template transaction by the register code you can make a screwed up split for testing, click Cancel and it will be saved anyway.
Comment 34 David Carlson 2016-01-26 16:23:50 UTC
I just ran across https://bugzilla.gnome.org/show_bug.cgi?id=729748 which documents that other bug.  I will now try that nightly build.
Comment 35 David Carlson 2016-01-26 18:38:35 UTC
That is weird.  In release ..."built from git rev 42e5dd5+ on 2016-01-26."
I tried editing a Scheduled Transaction by entering text into the next split line memo but not the account box.

I got a nice warning when I clicked OK.  I then clicked Cancel, which closed the editor.  Then I re-opened the editor on that transaction and found the memo still there with no account.  I again clicked Cancel and ran the SLR with the box to Review Created Transactions checked.  The transaction was created, but it had a mix of information which I had removed in a previous edit yesterday in the regular release 2.6.11 along with the information I had added in that edit but not the extra memo which I had added just a moment ago.  Also, the SLR did not request any value as when a formula needs one.  It did add an imbalance amount to balance out the amount that I had deleted yesterday.

Re-opening the SX Editor after SLR, the test memo had disappeared but the amount that had been deleted yesterday did not re-appear.  I could not repeat the experiment in that copy of the data because the transaction had been entered.

Now I am reloading the data file to try again.
Comment 36 John Ralls 2016-01-26 19:07:17 UTC
Weird indeed. If after reloading it still behaves the same please attach screenshots showing the editor (1) before you click Cancel and (2) after running the SLR as well as the SLR, so three in all.
Comment 37 David Carlson 2016-01-26 19:14:37 UTC
On the next try from what I thought was the same data file after I had closed without saving, I got slightly different results when I ran the SLR on the freshly corrupted transaction I got a new warning "Invalid Transactions" which named the bad transaction.  Nice! ;)  I clicked Close and found that transaction had not been created and the next date was still May 11.  This is acceptable to me.  

Now I have re-opened the transaction in the SX Editor and deleted the spurious memo.  I needed to move the curser up a line and hit enter to complete the removal and close the editor.

Then I re-ran the SLR and found that the transaction was created, but it still had the amount which was deleted previously in release 2.6.11 and was not present in the current editor display of the transaction data, along with an additional imbalance amount to balance the transaction.

I am now puzzled how to delete this invisible amount from the scheduled transaction.

I have a screen shot of the new warning but I will have to repeat the experiment to get the others.
Comment 38 John Ralls 2016-01-26 19:21:52 UTC
Can you find the SX in your data file and paste in the whole element? Maybe that will provide a clue about why it isn't showing up in the SX editor.
Comment 39 David Carlson 2016-01-26 20:13:41 UTC
Created attachment 319790 [details]
Bad SX before SLR

I needed to select an automatic backup from early this morning as GnuCash had done an auto save before I closed it.

The first screenshot shows the newly created corrupted transaction with the curser still entering the Test text.  If I click Cancel with the curser in that position, the text is not saved.  I need to move the curser up a line and click Enter before clicking Cancel to keep the spurious text.

Note that there is no amount in the second split line.  That is where an amount appears when the SLR runs.

Note in the SLR screenshot that I needed change the status from Reminder to To Create, but there is no request for an amount.

After the SLR rejected the transaction I re-opened the editor and removed the spurious text.  Again I had to move the curser up a line and click Enter before I could click OK to close the editor.  

This time the SLR accepted the transaction but it has an amount in the second split line which I had removed yesterday.  I may be able to fix that by putting a zero in that box, but I have not tried that yet.
Comment 40 David Carlson 2016-01-26 20:14:42 UTC
Created attachment 319791 [details]
SLR Dialog
Comment 41 David Carlson 2016-01-26 20:15:37 UTC
Created attachment 319792 [details]
Created transaction with extra amount
Comment 42 John Ralls 2016-01-26 22:38:41 UTC
Try entering a 0 in either the debit or credit formula for the PREAUTHORIZED CREDIT split. Some years ago Christian Stimming created a numeric value slot for formulas that didn't need variables as a way to get around the localization problem with storing floating point numbers as strings. It was kind of a hasty change and was the cause of another of the SX bugs. I bet it isn't getting cleared if you clear the formula.
Comment 43 Geert Janssens 2016-09-14 08:23:53 UTC
Still something to do here or can this bug be closed ?
Comment 44 John Ralls 2016-09-15 21:33:41 UTC
I *think* I addressed the last item with https://github.com/Gnucash/gnucash/commit/7a25e2a716b1cf0e518a854fd181cb7034ad0292. A test would be nice to confirm it.
Comment 45 Geert Janssens 2017-09-22 11:04:53 UTC
Given no more complaints, I assume this has indeed been fixed. If not feel free to reopen the bug.
Comment 46 John Ralls 2018-06-29 23:43:01 UTC
GnuCash bug tracking has moved to a new Bugzilla host. This bug has been copied to https://bugs.gnucash.org/show_bug.cgi?id=754856. Please update any external references or bookmarks.