GNOME Bugzilla – Bug 527192
Loading 5000 DAAP tracks is slow
Last modified: 2020-03-17 08:23:13 UTC
In Banshee trunk (r3726), loading my DAAP share with 5000 tracks is so slow that I haven't even let it run long enough to finish. Some rough benchmarking shows that I can do 100 tracks in 15 seconds, and 1000 tracks in 2 min 48 sec. I think it continues to get slower. In Rhythmbox my tracks are loaded and ready to play in about 3 seconds after clicking on the share. During the whole loading process, Banshee is pegging my dual-core CPU at ~60%. If I change DaapSource.OnClientUpdated to use "daapTrack.Save (false)" to prevent notifications, the load time is cut down to 19.25 seconds. Of course, then the view doesn't do live updating during the loading process. I haven't tried tracking down what all happens when notifications are on during a call to Save. I would guess that not saving to the database would also save quite a bit of time, though I understand the trade-off here. I also noticed that it takes about 3 seconds just to get from the time I click on my share (and Banshee begins connecting) to the time it starts loading the tracks (when DaapSource.OnClientUpdated is called).
(In reply to comment #0) > I also noticed that it takes about 3 seconds just to get from the time I click > on my share (and Banshee begins connecting) to the time it starts loading the > tracks (when DaapSource.OnClientUpdated is called). Of course, this is the time it takes to connect and get all the track information from the DAAP share. The time I was trying to optimize was loading that information in the Banshee source, I guess.
I can confirm this, DAAP shares take very long to load: for me, connecting to my test share (13k songs) takes about two hours. While this is in progress, the CPU is constantly at 65-75% (D2C 6600).
I've got a third on this. Considering my share is over 20k songs, it takes.. well.. a very long time. In stable, however, it takes considerably less time. Perhaps a minute. I noticed that trunk had similar performance, as well, before the tracks were sorted into the browser. Perhaps this is unrelated, but that's about the time I noticed that the performance went to pot.
Created attachment 113184 [details] [review] patch to only display updates every 100 tracks This patch fixes the problem for me by only updating the display every 100 tracks. The other 99 tracks are saved but no display updates are made.
Thank you for you patch. I can't comment on the merits of the patch, as I don't use DAAP. But I see that it has some formatting issues : spaces for indentation, variable name. You can have look at the HACKING file for details. Couldn't the trackCount variable be local to the OnClientUpdated method ? Saving memory, byte by byte... ;)
*** Bug 538169 has been marked as a duplicate of this bug. ***
With the new DAAP code in trunk (r4205), the loading time is much improved. Still slow compared to Rhythmbox. My 5124 tracks are loaded in 31 seconds (250 tracks at a time). I don't think the CPU usage is an issue. Updating bug summary to reflect this.
Well I have not yet tested the new DAAP plugin, is it save to test? The plugin should clearly not use as much CPU as it used to (which is probably fixed anyway now) but before it was an issue.
As far as I can tell, the code in trunk now does something similar to what is done in the patch, so making it as obsolete.
Now, tracks seem to be loaded 1250 at a time, and my tracks load in ~20 seconds. Still very slow compared to Rhythmbox. Now that we're so close to the 1.2 release, I'm confirming as NEW, and changing the target to 1.4.
I think the main slowness left is due to us not loading the N thousand tracks into the database within one or more SQL transactions, which blocks on adding thread-safe transactions back into HyenaSqliteConnection. There's a long note in that class explaining what needs to happen.
I have a patch that adds back transactions - makes the DAAP loading take 1/2 as long. You can grab it from my git repo using git clone http://banshee-project.org/~gburt/banshee.git
Although loading my DAAP share does not take as long as this report suggests, it still takes much too long. I have 7000 tracks and it takes about 3 minutes to load. The CPU on the machine running Banshee is 100% maxed during loading the share. Thus, this does not appear to be a network bottleneck. It looks like the algorithm to build the internal data structures cannot cupe with the amount of data.
Banshee is not under active development anymore and had its last code changes more than three years ago. Its codebase has been archived. Closing this report as WONTFIX as part of Bugzilla Housekeeping to reflect reality. Please feel free to reopen this ticket (or rather transfer the project to GNOME Gitlab, as GNOME Bugzilla is being shut down) if anyone takes the responsibility for active development again. See https://gitlab.gnome.org/Infrastructure/Infrastructure/issues/264 for more info.