I can second this. I'm using Windows 7 64-bit. The main places I have noticed the UI slowness are the account list and the reconcile window. Resizing columns is especially excruciating.

It seems much slower for me on Linux as well, but I'm not yet 100% certain what is causing the slowdown, because there appears to be an error in the packaging which is causing gnucash to attempt to recompile the Scheme files into /usr/share/gnucash (where it doesn't have write access) every time it starts up. Once I've figured out how to fix that I'll do some measurements and post here about relative performance of 2.4 vs. 2.6 on Linux.

I would not expect guile to attempt to store the compiled files in /usr/share/gnucash. The guile manual says the compiled files are normally stored in
$XDG_CACHE_HOME/guile/ccache [1]

You can disable autocompilation by setting environment variable
GUILE_AUTO_COMPILE=0

But this would not affect any Windows users. Only guile 2 compiles its source files. The windows build is still using guile 1.8 so the slowness must come from somewhere else.

@Ian: regarding the slow load time, what timings do you get if you start from an uncompressed data file ? I know we had to change the decompression code due to a bug in libxml.


[1] https://www.gnu.org/software/guile/manual/html_node/Environment-Variables.html#Environment-Variables

I use an uncompressed data file, and it's definitely significantly slower in 2.6 than 2.4.

Jonathan are you talking about load time ? My question was particularly regarding the part of the startup time that loads the file.

There are many aspects in this report and I'm trying to look at only one at the time.

I fixed the scheme compile issue. There was a permissions issue in ~/.cache/guile in my home directory.

In 2.4, startup up to "Loading user data" takes 7 seconds and loading the user data takes 4 seconds. In 2.6, startup up to "Loading user data" takes 8 seconds and loading the user data takes 7 seconds. Loading user data is consistently _much_ slower in 2.6 than 2.4.

I would like to update the synopsis and OS for this bug to indicate that it is not Windows-specific, but I can't figure out how to do that. Am I missing the bugzilla permission that would make it possible for me to do that or something?

Before you do that, do you also experience the choppy column resize behaviour in the account hierarchy ?

> @Ian: regarding the slow load time, what timings do you get if you start from
> an uncompressed data file ? I know we had to change the decompression code due
> to a bug in libxml.
> 

OK using a different (slower) machine with Windows 7 32 bit and an uncompressed data file (14.1MB now) I get the following:

2.4.13: 32s to 'Loading User Data' plus 26s = 58s total
2.6.0:  39s to 'Loading User Data' plus 83s = 122s total (the last 13s of the data loading the progress bar moved left and right rapidly, it didn't do this on 2.4.13).

So it doesn't look like the file compression is the problem.
On this machine the UI was even worse - resizing columns almost impossible, and even getting a menu to drop down from the top (e.g. File menu) took a couple of attempts. The CPU wasn't working hard during the UI slowness, maybe GPU issues (this netbook has built-in graphics...)

Switching from Accounts tab to Budget tab is also very slow, i.e. ~1.5s, rather than instantaneous.

>gnucash --nofile
6:2014/01/24 00-23-40:gwen(25640):gui.c:  125: Using own callbacks in gui 0x22e0e60
Found Finance::Quote version 1.18
>

but:

>gnucash
6:2014/01/23 21-24-48:gwen(6745):gui.c:  125: Using own callbacks in gui 0x1ceaf50
Found Finance::Quote version 1.18
java version "1.7.0_45"
OpenJDK Runtime Environment (IcedTea 2.4.3) (suse-24.9.1-x86_64)
OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)
>

I think loading a java engine could have some impact.

Undoubtedly, but loading the data file is taking almost twice as long and I can't see how that would have anything to do with the java engine.

(In reply to comment #11)
> >gnucash --nofile
> 6:2014/01/24 00-23-40:gwen(25640):gui.c:  125: Using own callbacks in gui
> 0x22e0e60
> Found Finance::Quote version 1.18
> >
> 
> but:
> 
> >gnucash
> 6:2014/01/23 21-24-48:gwen(6745):gui.c:  125: Using own callbacks in gui
> 0x1ceaf50
> Found Finance::Quote version 1.18
> java version "1.7.0_45"
> OpenJDK Runtime Environment (IcedTea 2.4.3) (suse-24.9.1-x86_64)
> OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)
> >
> 
> I think loading a java engine could have some impact.

I don't know why suse loads a java vm at gnucash startup. This is not the case in Fedora and definitely not something added by gnucash. GnuCash doesn't use java.

So the java issue should be investigated by the suse packagers

The other poor performance issues can be categorized in two:
- slower data file loading
- worse performance of treeview based gui elements (account hierarchy, budgets, reconcile,...).

I would like to split this into two bugs because it becomes very confusing very quickly if the two performance issues are discussed interleaved.

Since most of the comments on this bug focus on slower data file loading, let's keep it focused on that. I'll open another bug for the treeview issues.

For ui related slowness, please follow up on bug 722903. I have copied all ui performance related comments to that bug and cc'd the authors.

I have updated the title of this bug now. Please only add comments here related to the file load performance. Thank you.

It would be helpful if someone could spend some time to find out when this slowness started. For windows all the stable and unstable release installers can be found in http://code.gnucash.org/builds/win32/releases/

If someone can install older versions from this directory and check where the load times start going wrong that would really help narrowing the scope.

(In reply to comment #13)
> (In reply to comment #11)
> > I think loading a java engine could have some impact.
> 
> I don't know why suse loads a java vm at gnucash startup. This is not the case
> in Fedora and definitely not something added by gnucash. GnuCash doesn't use
> java.
> 
> So the java issue should be investigated by the suse packagers

from http://lists.gnucash.org/logs/2014/01/2014-01-24.html#T11:13:41:
<DimStar> fell_: gnucash uses webkit.. and webkit is stupid: it loads all plugins that are available...

So the first opened report starts webkit
and webkit loads the javaVM.

So this part of the thread belongs to Bug 722903

(In reply to comment #17)
> (In reply to comment #13)
> > (In reply to comment #11)
> > > I think loading a java engine could have some impact.
> > 
> > I don't know why suse loads a java vm at gnucash startup. This is not the case
> > in Fedora and definitely not something added by gnucash. GnuCash doesn't use
> > java.
> > 
> > So the java issue should be investigated by the suse packagers
> 
> from http://lists.gnucash.org/logs/2014/01/2014-01-24.html#T11:13:41:
> <DimStar> fell_: gnucash uses webkit.. and webkit is stupid: it loads all
> plugins that are available...
> 
> So the first opened report starts webkit
> and webkit loads the javaVM.
> 
> So this part of the thread belongs to Bug 722903

Frank, thanks for following up. I'm inclined however to ignore this part for a couple of reasons:
1. GnuCash 2.4 did use webkit already as well. Certainly on Windows. So it doesn't bring any additional slowdown between 2.4 and 2.6.
2. Loading the java vm is not something gnucash controls. That's something internal to webkit.

If there is a way we can disable webkit plugins ourselves then obviously my second reason is not correct. But then the requested change should be a completely independent feature request outside of the scope of performance comparisons between 2.4 and 2.6.

(In reply to comment #18)

Ok, created a separate RFE Bug 722926 - Can we control webkits plugin loads?

At least on Mac, there are two somewhat separate issues that are leading to the slower data file loading that I see.

With Gnucash 2.4.x, I see it get to the "Loading user data..." stage, then take about 16-20s to do that (with the progress bar going from left to right), then it shows the Gnucash window.

With 2.6, I see fairly similar behavior at first, but once the progress bar reaches the right edge (again, seems to be order of 20s or so) it starts bouncing back and forth like comment 9 says.  It keeps doing this for about 3.5 minutes in my case and then the Gnucash window shows.  During those 3.5 minutes Gnucash is only using about 30% of one core.

I tried doing a quick sample in Activity Monitor during those 3.5 minutes, and it showed almost all the CPU time being spent in GTK calls made from XML parser callbacks.  Presumably the intent is to keep the progress bar showing progress during the parse, but the result is that almost all the time is spent doing UI updates and not much time is spent actually parsing the XML.  This is the first issue.

For the second issue, I tried just clicking on the splashscreen once it gets to the "bouncing back and forth" stage.  This makes the splashscreen go away, and Gnucash's CPU usage goes up from 30% to 100% of one core.  Sampling now shows no GTK calls, just XML parsing, conversion of DOM bits to transactions, etc.  The Gnucash window is shown about 30s later, which is a lot better than 3.5 minutes, but a lot worse than the "immediately" of 2.4.x...

Oh, and needless to say, while 16-20s to start Gnucash 2.4 is bearable, the 45-50s in 2.6 with the splashscreen hidden is a bit annoying and the 3.5 minutes is just unusable (in fact, it was only by accident that I ended up waiting that long for it to come up in the first place the first time; I thought it had hung when it started doing the back-and-forth thing and was about to kill it, when I had to go deal with something else for a bit and came back to see a Gnucash window; then I started timing things).

I'd be happy to attach the activity monitor samples for 2.6 both with the splashscreen up and without the splashscreen, if people want...  Just let me know!

Profiling shows that data file loading is slowed down by checking every input string for being valid XML and UTF8, and substituting a valid character for the bad one.

I've pushed 6461b3, which speeds up the string check by about 50%, but it's still many times slower than 2.4.

I'm using 2.6.2 with a 2MB compressed data file on both windows xp and windows 7.  It takes 4 minutes to load on windows xp and 70 seconds on windows 7 (which is a 3 cpu machine).  I observed, when the program is loading data, it never uses more than 30% of one cpu and the memory usage remains fairly constant.  Also, disk usage is periodic peaks.  So it appears there is some cap on the CPU usage (scheduling issue) because the cpu is doing work all the time, but never makes use of the resources that are available.  If this was working correctly, I should be getting my data to load in about 25 seconds on one CPU or 10 seconds if there was multi-cpu support...

Is the application running running in the GUI glass or something?  I find this a major issue...my xp machine runs quicken fine and I want to use gnucash, but I can't.  Its even painful to wait on my really fast windows 7 machine.  Can you bump the priority on this?

So would it be possible to not do the progress bar updates during the XML parse?  Or to throttle them or something?  That seems like it would help a good bit, given comment 20...

(In reply to comment #25)
> So would it be possible to not do the progress bar updates during the XML
> parse?  Or to throttle them or something?  That seems like it would help a good
> bit, given comment 20...

Try unchecking "show the splash screen" in Preferences>General.

I've noticed a huge speedup in both load times and processing import account assignments since upgrading to OSX Yosemite, and 2.6.5 will load much faster on platforms with Guile 2 (which includes OSX) thanks to precompiling all of the guile code.

I recently installed 2.6.5 (rev 23d0f79+ on 2014-12-19) on my Windows 7 machine (the one used in comment 1) and things seem greatly improved.
Total load time is now ~20s instead of 52s on 2.6.0 from comment 1.
My data file is now 1,359KB compressed compared with 1,041 KB then.

I tested this with both the splash screen on and off, and it made no difference. The backwards-forwards behaviour didn't appear.

The UI seems better too.

(In reply to comment #27)
> I recently installed 2.6.5 (rev 23d0f79+ on 2014-12-19) on my Windows 7 machine
> (the one used in comment 1) and things seem greatly improved.
> Total load time is now ~20s instead of 52s on 2.6.0 from comment 1.
> My data file is now 1,359KB compressed compared with 1,041 KB then.
> 
> I tested this with both the splash screen on and off, and it made no
> difference. The backwards-forwards behaviour didn't appear.
> 
> The UI seems better too.

Sorry, I just realised the machine from comment 1 was Windows XP, so the comparisons are not valid. My current Windows 7 machine is much faster.
However things do seem improved on this machine with 2.6.5 compared to earlier versions.

> Try unchecking "show the splash screen" in Preferences>General.

That has the undesirable side-effect that it also doesn't show report progress for the reports that get generated at startup.  Which is mostly a problem because every so often someone changes one of those reports to basically hang on anything resembling a large data set, and when that happens it's nice to know which report got broken.

In any case, I just did a side-by-side test of startup times for 2.6.5 and 2.4.15 (on Mavericks, not Yosemite, in case that really matters).

With splash screen:
2.4.15: 35s
2.6.5: 384s (yes, 10x longer).

Without splash screen:
2.4.15: 28s
2.6.5: 47s

The splash screen time is, again, due to updating the splash screen UI for every single XML element in the data file.  The non-splash-screen time is ... I don't know, but waiting close to a minute for modern software to show anything at all you can interact with is pretty unusual, and the few other bits of software that do that show splash screens so you don't start wondering what lala-land they've gone off to.  :(

FWIW I've noticed a substantial speedup since upgrading to Yosemite. That aside,
it's not really updating for every XML element, just every commodity, account, and transaction. That's still plenty, and I agree that we could dial back on the user entertainment a bit.

As I pointed out in comment 22, most of the rest of the difference with 2.4 is from the valid unicode check on every string. We added that because if invalid UTF8 gets in to the database it can cause data loss. See bug 710823 and bug 7120824.

Hello everyone. Is the performance of 2.6.x still an issue for people? Can I assume that judging from the lack of recent updates on this issue that it has become overtaken by events?  If I'm wrong, perhaps I can help to facilitate this being addressed? If so, please send a contact request form in at https://patchpitch.com/contact
Cheers,
Paul.

> Is the performance of 2.6.x still an issue for people? 

Yes.  Just to make sure I just redid my timings from comment 29 (on Yosemite this time, and newer hardware).

With splash screen:
2.4.15: 17s
2.6.13: 100s

Without splash screen:
2.4.15: 15s
2.6.13: 32s

That's without any reports being generated on startup; I don't really have the time it would take to make those measurements with 2.6.x right now.  As a result, I've been staying on 2.4.x because 2.6.x performance is so abysmal that it's not usable.  Note that there has definitely been _some_ progress either from Mavericks to Yosemite or from 2.6.5 to 2.6.13 for the splashscreen case; now it's only 6x slower than 2.4.15, not 10x slower.  It's still really slow, though.

The lack of recent updates is just to do with everything having been said already.  There are several identified causes of the performance problems and  none of them are being addressed...

Thanks for the update, Boris!  I guess the question now is how much of a pain this is to everyone and how many people want this fixed in order to make use of the new features in 2.6.13.  I assume you'd be one vote :) ?

I have recently switched from running GnuCash 2.6.13 on Windows 7 or Windows 10 to running 2.6.11 on Debian Jessie Linux.  There, even with an additional lag caused by storing the data file on a NAS, load times are far more tolerable.  

My data file is about 5.6 MB compressed.  I think that it takes about 20 to 30 seconds to open in Linux with a couple of reports.

In fact, I am now usually using RAS to run the Linux machine from a Windows 10 laptop instead of opening the file in Windows.  A lot more complicated than it should be, but it works.

GnuCash bug tracking has moved to a new Bugzilla host. The new URL for this bug is https://bugs.gnucash.org/show_bug.cgi?id=721822. Please continue processing the bug there and please update any external references or bookmarks.