GNOME Bugzilla – Bug 651425
All available resources consumed when importing
Last modified: 2020-03-17 09:30:11 UTC
Whenever I import more than 2,000 songs into my library, my entire system slows to a near-halt. 99% of my 8gb of RAM is consumed, as is 99% of my 16gb swap. All of my cpu cores are maxed out as well. I've tried importing a few hundred songs at a time, but as soon as I breach the 2,000 mark in a single session, the system freezes again. I then have to hold the power button down on my pc to power it off, as it is impossible to navigate to shut it down from within the system (due to it being fully consumed by Banshee). Upon restart my library will be there, and Banshee is fully usable, but if I try and import more than 2,000 songs again, the process is repeated. My library is comprised of roughly 10,000 songs. System: Ubuntu 11.04 64bit Intel Core i7 8gb RAM 16gb Swap Banshee 2.0.1 If any more information is needed, let me know, and I'll try and provide it.
Created attachment 191454 [details] banshee debug log I am having the same problem on a Gentoo box, using KDE 4.6.4 as my desktop environment. To reproduce: 1) cd ~ 2) rm -rf .gnome* .gconf* .config/.banshee* 3) banshee --debug 2>&1 | tee banshee-debug-log.txt 4) Inside Banshee: Import music collection (about 4GB of M4A and MP3) Before step #4, Banshee is very responsive. However, within a few seconds of launching folder import, banshees quickly swells to consuming 10GB of RAM, limited only by my HDD's ability to swap back and forth. The system begins to thrash violently and can only be recovered using "killall banshee" in a terminal. During this thrashing, importing completely stalls. 1 song is imported every minute or so, at best. The out-of-control memory consumption, swapping, and thrashing kills everything. If I kill banshee and re-open, it quickly resumes this behavior and is completely unusable. I also have this problem if I try to import a CD into a blank install, so I think the problem is way bigger than importing of folders. Maybe I'm missing some GNOME utility, since I'm running on KDE and have little to no GNOME apps running?
Created attachment 191461 [details] banshee database debug log I tried the latest version of banshee, 2.1.0, and I had the same problem. I'm attaching the fuller debug log with database debug too. To refine the problem statement for me: I actually *only* see the huge memory runaway during import. If I close banshee and re-open it, it responds great - until I do another import. :sigh: Curiously, the 2.1.0 version died after poking around on the various plug-ins. Specifically, it seg-faulted (SIGSEGV) after clicking on the Amazon store applet. Maybe I have some old library versions? Incidentally I have created a Gentoo ebuild and am submitting it. If you help me straighten out my libraries, it might help an entire distro, as well as possibly resolving this issue. Thanks!
link to Gentoo build request: http://bugs.gentoo.org/show_bug.cgi?id=374357
Gabriel, Trevor, thanks for all the information. It is possible that this bug is a duplicate of bug 648592. That bug has a patch attached to it, would it be possible if you guys could test that patch to check if it fixes your problems? Thanks very much.
Created attachment 191474 [details] banshee 2.0.1 database debug log with patch from bug # 648592 I applied the patch to 2.0.1, but it did not make any difference that I could see. :( Banshee may have topped out a little slower, but not much. FWIW, the real problem seems like a severe memory leak. I don't know how else banshee could climb to 10 GB of consumed RAM+SWAP by importing less than 100 songs, limited only by system thrash. ... new debug log is attached. ... Thanks! :)
Thanks.
Trevor, you need to change the dbus dependencies for your ebuild. Banshee now requires dbus-sharp >= 0.7 and dbus-sharp-glib >= 0.5 (the ndesk-dbus dependencies were dropped in 2.1.0). I've noticed memory leaks on import, but nothing as dramatic as you have. After importing a few thousand media files, memory uses will be up (and stay up) by 100-150MB. It doesn't sound like you're having the same problem as bug 648592, as that is more of an issue of database optimization after a large number changes to the database (due to importing many, many files). Though Gabriel's problem sounds much more like that bug.
Created attachment 191526 [details] banshee 2.1 debug log based on updated ebuild, library dependencies Mackenan, thanks for the feedback. I have updated the 2.1.0 ebuild and refreshed the downstream bug. I also included the patch that Andrés mentioned. Unfortunately, it did not help me. I already had those two libraries installed, even though they were not strictly listed in the dependencies. I have attached the new debug log. Also, banshee again crashed when trying to launch an app, miro, in this case. Do you see any other outdated dependencies? Should we still be using the patch from 1.7.4 to make web-kit optional? ... epatch "${FILESDIR}/${PN}-1.7.4-make-webkit-optional.patch" # upstream bug 628518 ... Just for reference, here is how "top" shows the state of my system during import: top - 10:36:12 up 2:17, 6 users, load average: 4.88, 2.38, 1.29 Tasks: 171 total, 1 running, 170 sleeping, 0 stopped, 0 zombie Cpu(s): 9.3%us, 1.3%sy, 0.0%ni, 23.3%id, 65.7%wa, 0.0%hi, 0.5%si, 0.0%st Mem: 6112388k total, 6016056k used, 96332k free, 3964k buffers Swap: 16777212k total, 4795892k used, 11981320k free, 54652k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1663 tbowen 20 0 9793m 5.2g 5128 D 4 88.8 0:46.16 banshee 4370 tbowen 20 0 845m 99m 7800 S 1 1.7 1:15.37 thunderbird-bin 4312 tbowen 20 0 2017m 56m 3452 S 0 0.9 0:21.48 java 3940 tbowen 20 0 700m 39m 14m S 4 0.7 2:52.97 kwin ... Thanks!
Removing the 1.7.4 patch from the build did not make any difference. Import still blew up to almost 10 GB by the time I had imported 80 songs.
Looking at your first debug log, I see a number of error messages like the following: [11 Debug 08:30:03.243] Encountered a problem processing: file:///winxp/Documents%20and%20Settings/Bowen/My%20Documents/My%20Music/iTunes/iTunes%20Music/Shinedown/The%20Sound%20of%20Madness%20(Bonus%20Track%20Versio/14%20Son%20of%20Sam%20(Bonus%20Track).m4a [11 Warn 08:30:03.260] Caught an exception - System.ArgumentException: Length must be non-negative Parameter name: length (in `taglib-sharp') at TagLib.File.ReadBlock (Int32 length) [0x00048] in /var/tmp/portage/dev-dotnet/taglib-sharp-2.0.3.7/work/taglib-sharp-2.0.3.7/src/TagLib/File.cs:615 I'm not sure if that's the source of your problem or not. Some of your m4a files seem to be causing a problem with taglib-sharp. As far as I can tell, the length parameter is most likely receiving a negative value as a result of casting a long value into an int (this happens in TagLib.Mpeg4.Box.DataSize, where it is calculated as header.DataSize (a long) plus data_position (also a long) minus DataPosition (again, a long) and cast into an int). I'm not sure if this is in any way connected to your memory leak.
You are probably right that my songs' metadata have been somehow corrupted. I have experimented with all of the Linux music players, trying to find something to replace iTunes. Songbird has previously been my personal favorite, but they have dropped Linux support. No dobut, the metadata was corrupted during all that experimenting. Anyway, can you recommend any tool, method, or solution for cleaning up all of the metadata? I'd like to save what I have (artist info, dates, etc.), but I don't want to propagate what seems to be corrupt metadata. Then, I can at least eliminate that variable from further consideration in this issue. Thanks!
Banshee is not under active development anymore and had its last code changes more than three years ago. Its codebase has been archived. Closing this report as WONTFIX as part of Bugzilla Housekeeping to reflect reality. Please feel free to reopen this ticket (or rather transfer the project to GNOME Gitlab, as GNOME Bugzilla is being shut down) if anyone takes the responsibility for active development again. See https://gitlab.gnome.org/Infrastructure/Infrastructure/issues/264 for more info.