After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 421376 - [Patch] Automatically scan music folders for new songs
[Patch] Automatically scan music folders for new songs
Status: RESOLVED DUPLICATE of bug 385965
Product: banshee
Classification: Other
Component: Importing
git master
Other All
: High enhancement
: 1.x
Assigned To: Banshee Maintainers
Banshee Maintainers
: 455688 497992 525079 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2007-03-22 06:56 UTC by Scott Peterson
Modified: 2012-07-20 14:36 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
A patch to add auto-scanning of music folders for new songs (21.25 KB, patch)
2007-03-22 06:59 UTC, Scott Peterson
none Details | Review
A revised patch (21.39 KB, patch)
2007-03-23 06:43 UTC, Scott Peterson
none Details | Review
A revision to handle child directories (23.69 KB, patch)
2007-03-25 04:34 UTC, Scott Peterson
none Details | Review
Now with FileSystemWatchers! (24.74 KB, patch)
2007-03-27 05:18 UTC, Scott Peterson
none Details | Review
A slightly beefed up version of the patch (24.24 KB, patch)
2007-03-29 06:55 UTC, Scott Peterson
none Details | Review

Description Scott Peterson 2007-03-22 06:56:33 UTC
This patch adds a new option to the Banshee Preferences: "Watch music folders for new songs" (off by default). A list is maintained of all music folders - these includes the library folder and any folders imported with the Local Folder Importer. At a given interval (configurable in gconf, default is 10 seconds), a worker thread will recursively check the DirectoryInfo.LastWriteTime for all subfolders of all registered music folders. If any folder shows new activity, an import will be queued. The import is done through the new ImportManager.UpdateInstance. This instance will not display the standard import ActionUserEvent widget. If any new songs are successfully imported, a special ActionUserEvent will briefly appear to notify the user.

The directory walk is fast, the scan interval is large, and new files are rare, so performance is very good.
Comment 1 Scott Peterson 2007-03-22 06:59:37 UTC
Created attachment 85097 [details] [review]
A patch to add auto-scanning of music folders for new songs

My local copy of the source is significantly different from SVN so I had to hand-prune this diff file. I hope I didn't make any mistakes.
Comment 2 Scott Peterson 2007-03-23 06:43:04 UTC
Created attachment 85162 [details] [review]
A revised patch

This patch delays allocation of ImportManager.UpdateInstance until there are actually updates to perform. Since updates are rarely performed, I considered making UpdateInstance a WeakReference after an update is performed, but I think the memory gains are negligible.
Comment 3 Scott Peterson 2007-03-25 04:34:10 UTC
Created attachment 85244 [details] [review]
A revision to handle child directories

If one directory in the scan list is a subdirectory of another, it will not be scanned independently since it will get scanned while scanning the parent directory. This patch also ensures that imported folders are always scanned. I also lowered the priority of the scanning thread to Lowest.
Comment 4 Ruben Vermeersch 2007-03-26 12:26:40 UTC
We used to have an inotify plugin that did this. Can't we register inotify watches after the first walk? This would reduce the need to walk all directories again.
Comment 5 Scott Peterson 2007-03-26 19:52:14 UTC
Is there an inotify equivalent on Windows?
Comment 6 Scott Peterson 2007-03-27 05:18:40 UTC
Created attachment 85366 [details] [review]
Now with FileSystemWatchers!

Wow, who knew there was a FileSystemWatcher class ;)

There are two gotchas:
1) The FileSystemWatchers are listening for a Create event. When songs are copied in to a watched folder, the Create event fires immediately and then it's a race between the filesystem and the managed code. Sometimes the import code executes before the file is done copying. Havoc ensues! While instantiating the LibraryTrackInfo object, an array out of bounds error flies from the bowels of the IO system since the file size changes mid-read. My solution is to catch that error and goto the try block again. That is The Original Lame but I can't find another way of ensuring that copying is complete (short of testing the file size, waiting, and checking it again, which is equally lame and delays the import of otherwise fine files). Curses!

2) Due to the FileSystemWatcher's true IncludeSubdirectories, if you're watching an NTFS directory (as I am) which has some non-UTF8 encoded directory paths (as mine does) and you run from a terminal (as I do) then Mono emits anywhere between some and a metric butt-load of "Bad encoding" messages. Minor but annoying.
Comment 7 Scott Peterson 2007-03-29 06:55:08 UTC
Created attachment 85501 [details] [review]
A slightly beefed up version of the patch

This patch adds fault tolerance for the FileSystemWatchers (if a network connection providing access to a directory of interest is interrupted) and gets rid of a heinous goto statement.

But there is a problem! As I use the patch, I find that the FileSystemWatchers fail to report new files after some (unknown) period of time. I haven't had time to fully investigate the matter, but I know that they do not throw an Error event. They simply seem to die. A cursory Google search reveals no answers. Does anyone have some insight into this?

If anyone has time to test this, that would be wonderful. I've observed the problem after leaving Banshee open for roughly an hour but it could occur anytime between two and sixty minutes past the FileSystemWatcher instantiation. The first step is determining when things go pear shaped.
Comment 8 Josiah Ritchie - flickerfly 2007-08-24 02:12:42 UTC
*** Bug 455688 has been marked as a duplicate of this bug. ***
Comment 9 Andrew Conkling 2007-10-13 03:46:51 UTC
I wanted to check this out, but in its current state it doesn't patch in SVN. I wish I could discern where to do that myself, but could the patch be updated? (It seems to only be an issue in src/Core/Banshee.Base/Gui/PreferencesDialog.cs.)
Comment 10 Gabriel Burt 2007-10-24 20:36:40 UTC
Scott, are you seeing anything interesting RAM-usage wise?  There are some open bugs with FSW (which is why I've held off on using it for Banshee/F-Spot):

https://bugzilla.novell.com/buglist.cgi?query_format=specific&order=relevance+desc&bug_status=__open__&product=Mono%3A+Class+Libraries+&content=filesystemwatcher

Thanks!
Comment 11 Scott Peterson 2007-10-24 21:01:48 UTC
Andrew, I may look into updating this patch later but I'm afraid I won't be able to get around to it for some time. If you or someone else would like to do the work be my guest, but beware that I wrote this code when I was young and stupid so it's pretty ugly. If you don't mind waiting, I'd like to completely re-write this when I have the chance.

Gabe, I never really looked into what was up with FSW, but it does indeed seem that something is. When I get around to this patch, I'd be willing to poke around in FSW and look at addressing those bugs.
Comment 12 Andrew Conkling 2007-10-24 21:40:05 UTC
(In reply to comment #11)
> Andrew, I may look into updating this patch later but I'm afraid I won't be
> able to get around to it for some time. If you or someone else would like to do
> the work be my guest, but beware that I wrote this code when I was young and
> stupid so it's pretty ugly. If you don't mind waiting, I'd like to completely
> re-write this when I have the chance.

Well, this is one of the things I'd really like to see in Banshee, but unfortunatly my C#/Mono skills are not up to par. I'm working on that, but it is taking time of course. :)
Comment 13 Andrew Conkling 2007-11-19 14:59:26 UTC
Bug #497992 is a duplicate of this.
Comment 14 Gabriel Burt 2007-11-19 16:44:57 UTC
*** Bug 497992 has been marked as a duplicate of this bug. ***
Comment 15 Gabriel Burt 2008-05-19 03:53:50 UTC
Changing to target the 1.x series.
Comment 16 Andrew Conkling 2008-06-17 11:40:14 UTC
*** Bug 525079 has been marked as a duplicate of this bug. ***
Comment 17 Bertrand Lorentz 2008-07-01 15:10:33 UTC
I think this one should be marked as a duplicate of bug #385965, as the patches here are obsolete and there's a recent one in bug #385965.
Comment 18 Andrew Conkling 2008-07-10 19:35:28 UTC
Thanks Bertrand; marking.


*** This bug has been marked as a duplicate of 385965 ***