I'm not sure if this helps, but Muine already has inotify support - perhaps some code could be lifted from it?

I would also like to see this feature added :-)

Let me add that IMHO this is a key feature in music players. It's common for people to have a single folder where all the music resides (in their respective folders, of course), and it's great when the music player does all the work of detecting when files are added and deleted. This should be done both on startup and during runtime. I think Muine does this the best, from all GNOME music players.

Created attachment 95839 [details]
rhythmbox-library_watch.png

Maybe something like "Watch my library for new files" could be interesting in banshee (already available in rhythmbox)...

Thanks :-)

I would like to see this feature added as well. Rhythmbox monitors the assigned music folder very well, but I would much prefer to use Banshee over Rhythmbox...

By the way, older versions of Banshee support this feature via FAM plugin but newer don't. Why? Please add an inotify support!

*** Bug 486642 has been marked as a duplicate of this bug. ***

Subscribed!

Simone, you can subscribe without saying anything :) .
Just check the option and click "submit".

*** Bug 530739 has been marked as a duplicate of this bug. ***

Before I filed my bug which was just marked as a duplicate, I searched for an already existing bug report. I didn't find this one because I was looking for bugs in versions 0.98.x and SVN trunk. So could this bug's version please be changed to SVN trunk, because this feature request has yet to be implemented?

*** Bug 532358 has been marked as a duplicate of this bug. ***

I hear that tangerine and muine had this implemented pretty nicely.   Should probably at least look at their code and see if we can mimick it.

Thanks Gabriel. I'll have a check of their code.
I do already have an idea on how I'll implement it, though. :)

Honestly banshee would be perfect if it could watch folders and also update the metadata, it's really frustrating that it doesn't even update the metadata when i re-import the folder (without deleting the contents of the library, that is).  From the top of my head 3 well known players already support this feature, rhythmbox,exaile and amarok.

PLEASE have this up by the next release.

(In reply to comment #3)
> Let me add that IMHO this is a key feature in music players. It's common for
> people to have a single folder where all the music resides (in their respective
> folders, of course), and it's great when the music player does all the work of
> detecting when files are added and deleted. This should be done both on startup
> and during runtime. I think Muine does this the best, from all GNOME music
> players.
> 

I echo the same words.  "Let me add that IMHO this is a key feature in music players."

Mohamed, as defined by the target milestone for this bug, we're targetting 1.x. This extension will more than likey ship in 1.1 (1.0 has already been branched, tarballs generated, and packaged by most maintainers).

It will be added when we feel it is ready and relatively stable.
Expect a patch soon from me, though. It works nicely, though, it's still missing metadata refresh if the file is modified from an external application.

Gabriel, code review next week (after we ship)? :)

it seems that banshee 1.0.0 was released - packages for openSUSE 10.3 and 11.0 are ready in banshee (not banshee:preview anymore) repository.

It may be interresting to use tracker or beagle as backend, as they are designed to provide such features.  

Created attachment 113019 [details] [review]
First attempt at LibraryWatcher extension.

Attached is my version of the LibraryWatcher extension (and small patch to some of the Banshee DatabaseTrackInfo stuff). It imports and removes files from both music and video libraries in realtime.

At the moment, the only unimplemented feature is refreshing track metadata if it's edited from an external application (tag editors, another music player, etc).

*** Bug 539935 has been marked as a duplicate of this bug. ***

*** Bug 421376 has been marked as a duplicate of this bug. ***

Whats the progress of this enchancement?

Does this patch updata metadata aswell?

This patch is outdated.

Mohamed: Yes, if you edit metadata externally it should update.

I haven't updated this patch to work against trunk because apparently Aaron is working on a similar plugin (according to Gabriel), and, well, it's been about two months without a code review. I don't particularly feel like updating a patch every 2 weeks for something that isn't going to get applied. :)

If somebody wants to fix it up, feel free! It shouldn't be too difficult, I don't think.

Also, could someone poke Aaron about the status of his version of this extension?

(In reply to comment #26)
> Mohamed: Yes, if you edit metadata externally it should update.
> 
> I haven't updated this patch to work against trunk because apparently Aaron is
> working on a similar plugin (according to Gabriel), and, well, it's been about
> two months without a code review. I don't particularly feel like updating a
> patch every 2 weeks for something that isn't going to get applied. :)
> 
> If somebody wants to fix it up, feel free! It shouldn't be too difficult, I
> don't think.
> 

Just for confirmation assume I've got song.mp3 with artist="banshee", that has been added to the library.  If I change the artist name (or any tag for that matter) externally to say,artist="gnome", banshee will detect it right?  I'm asking because i've heard else where that the updating the metadata will be handled in a different patch.

Thanks

Aaron offloaded a patch to me about this. I've been really busy of late and haven't had time to finished it. I'll try to do this before the weekend, but if I don't get around to it, maybe someone else should take the torch.

AFAIK, this si a very much wanted enchancement, so I think a lot of people would be happy if some took the torch and decided to finish it so it can appear in 1.4.

(In reply to comment #28)
> (In reply to comment #26)
> > Mohamed: Yes, if you edit metadata externally it should update.
> > 
> > [snip]
> 
> Just for confirmation assume I've got song.mp3 with artist="banshee", that has
> been added to the library.  If I change the artist name (or any tag for that
> matter) externally to say,artist="gnome", banshee will detect it right?  I'm
> asking because i've heard else where that the updating the metadata will be
> handled in a different patch.
> 
> Thanks
> 

Erk, my bad. Looking at the code, I don't appear to have implemented this. Shouldn't be too difficult either.

Scott, any chance you could post the patch somewhere public anyhoo? More eyeballs; plus, it means less work for you. :)

*** Bug 547814 has been marked as a duplicate of this bug. ***

As implied in bug 547814, care should definitely be taken for moved files and their metadata (especially ratings and play count, as those cannot be written back to the file).

That's a current limitation of Rhythmbox's library monitoring (if you rename a file, you reset your play count and lose your rating). It would be nice to address this somehow.

As of today there is a MetadataHash column in the database (and atm protected property on DatabaseTrackInfo) that could be useful in identifying a track even if it's moved.  It's a MD5 hash of the title/artist/album/genre/duration/year.  I plan on using it for device syncing, but might be useful for this too.

Andrew, Gabriel; at the moment, the current logic for dealing with moved files is to simply update the path column in the database. This means all the other data relating to that file is left the same.

Any update?

Only zilions of new CCs

Could Aaron or Gabriel please give us a status update? :)
Optionally, some sort of mention in the next PPP roundup. I'd like to get some progress moving on this bug; I personally don't care whose code we end up using, as long as we're actively moving forwards. I'm happy to cleanup my patch and commit if that's what we want.

I guess this bug [1] is also blocking this feature.
[1] https://bugzilla.novell.com/show_bug.cgi?id=425468

This is in the recently released Banshee 1.3.0 development release: http://download.banshee-project.org/banshee/banshee-1-1.3.0.news

(In reply to comment #40)
> This is in the recently released Banshee 1.3.0 development release:
> http://download.banshee-project.org/banshee/banshee-1-1.3.0.news
 
Only partially.  What's in the release is a way to manually trigger a rescan (see Tools->Rescan).  Having it triggered by inotify is still TODO.

*** Bug 554968 has been marked as a duplicate of this bug. ***

*** Bug 556023 has been marked as a duplicate of this bug. ***

Any news on this MUCH needed feature?  Freeze is coming and the 1.4 release seems only a few weeks away... can I just reiterate that this feature is really important for the adoption of Banshee in the wider community.  In my experience, this is the missing feature which is the used the most when arguing against the use of Banshee (as default in Ubuntu for example).

Keep up the good work

Two years and two moths later... new version released (1.4.2) but still missing this feature.
It's the only (but the biggest) thing stopping me to leave Rhythmbox.

Bumping the target; 1.2 has come and gone.
For what it's worth, the benefit of Banshee's functionality is that Rhythmbox, when updating on startup, can take a lot of time and CPU/disk activity, so you can't always play your music right away.
I'd love to see this feature too, but it's not without its own consequences.
Alex, should this still be assigned to you? Looks like you were awaiting feedback from Aaron and Gabriel in comment 38?

+1
this is a must feature!! :)

> For what it's worth, the benefit of Banshee's functionality is that Rhythmbox,
> when updating on startup, can take a lot of time and CPU/disk activity, so you
> can't always play your music right away.
 
What's waiting 10s if you can get all your added songs! :)

Also, I am refering to a realtime feature! Like when you drag/drop a song in WMP (yeah yeah...buggy...bla..bla...bla lol)! :)

I should probably mention the only overhead that'd be caused by this is the time to register the inotify watch. From memory, Rhythmbox actually rescans all the files in its library at startup to see if they're all still there (and update it's metadata?), then does the inotify stuff. So, yes, that's why it takes a bit longer.

Andrew, yes, still awaiting feedback.

yeah i started using banshee but without this feature im afraid ill have to go back to rythmbox

(In reply to comment #48)
> > For what it's worth, the benefit of Banshee's functionality is that Rhythmbox,
> > when updating on startup, can take a lot of time and CPU/disk activity, so you
> > can't always play your music right away.
> 
> What's waiting 10s if you can get all your added songs! :)

I don't know; the Rescan action as currently implemented in Banshee takes a lot longer than 10s for me.... :)

Also, with the job scheduler being implemented in Banshee (http://gburt.blogspot.com/2009/04/putting-banshee-to-work.html) this could theoretically be implemented with less impact on the UI while it runs. I'm not sure about that, as it'd still be a big design decision, but perhaps that would make it more likely to be implemented.

well, it's too bad but I removed Banshee and now use Rhythmbox, which can play a song that you just downloaded without having to close it or rescan! ;)

I restart working on the banshee library watcher, a plugin to auto-update music & video library using Inotify.

Take a look at http://code.google.com/p/bansheelibrarywatcher/ or download it from svn: 
svn checkout http://bansheelibrarywatcher.googlecode.com/svn/trunk/ bansheelibrarywatcher-read-only  

The plugin is yet a bit slow and I'm working to speed it up. Test/bug reports/suggestions are welcome. (contact me at christian.martellini@gmail.com)

Cheers

martellc: in my honest opinion, it'd probably be faster and easier just to clean up the last patch I posted, instead of duplicating effort. But whatever you feel is easier. :)

Alexander:I saw your patch marked as obsolete so I thought it didn't worth to work on it. My mistake.
I wouldn't discuss of duplicating effort, I think that clean my old plugin or clean your patch are both easier the same..Anyway I will take a look at your patch :)
Thanks for your prompt answer.

*** Bug 581481 has been marked as a duplicate of this bug. ***

While implementing this it would be great to try to get right a thing or two that competitors such as rhythmbox don't. The main issue rb has is that if you start a music torrent, most of the files won't get picked up until a restart since parts of them (particularly the first I imagine) are invalid. In cases where a new file appears but isn't valid, it might be good to store in memory when this happened, and if the file is later modified after this, give it another shot (with some minimum timeout before a second try).

Ubuntu is considering adopting Banshee as the default media player but I suspect this issue will be one of the main points of contention, as others have suggested, as users are already used to having this functionality and would as such consider this a major regression.

Anyway, keep up the great work!

Created attachment 134881 [details] [review]
Second attempt at LibraryWatcher extension

This is a second attempt to a LibraryWatcher extension for banshee. I merged the old code from Alex Hixon first attempt with my Banshee Watcher plugin.. I think It's a good results... It works good with both 1.4.3 and 1.5.0 version.
If you don't like patchs you can download it from svn:
svn checkout http://bansheelibrarywatcher.googlecode.com/svn/trunk/ bansheelibrarywatcher-read-only

Cheers,
Christian

Created attachment 134882 [details] [review]
Second attempt at LibraryWatcher extension (patch without Makefile)

Thanks for your work, guys, I appreciate it.  Here are a bunch of nitpicks:

Put a comma between your names in the addin.xml

Consistently use 4 spaces for tabs.

There are lines like this -
"private   bool   forceUpdate;"
- please use just a single space instead of multiple spaces between keywords/etc.

Please put a single space between a method name and its arg list, eg this () not this().  See HACKING for the full style guide.

Please put copyright headers at the top of all .cs files.

Instead of doing a thread Abort () to stop the watcher, it would probably be better to just exit the watcher() method loop.  Also, rename the watcher method to Watcher.

Please update to work against 1.5/git master only - we no longer use relative URIs at all in CoreTracks, so I don't think it's possible or at least reasonable to try to support both.

"Lybrary Watcher Plugin" => typo

Thanks!

Created attachment 135006 [details] [review]
Library Watcher extension for git repository

- Updated library watcher extension to work with banshee git revision and to be compliant with banshee style guide.

Thanks for updating the patch !

Another round of nitpicks : ;)
- Don't include Banshee.LibraryWatcher.pidb in the patch, it causes an error and applying the patch fails.

- There are a couple of tabs left, in the beginning of the LibraryWatcherService class, and other places.

- In the build system files, try to add lines in the proper place when things are sorted alphabetically : configure.ac, build.environment.mk, src/Extensions/Makefile.am

- The "Lybrary Watcher Plugin" typo is still there

- Remove the "using Banshee;" lines, they're useless

- Remove the "Log.Debug ..." call in IExtensionService.Initialize, the extension startup is logged by the ServiceManager.

- Same thing for the try/catch in IExtensionService.Initialize : the ServiceManager catches and logs any exception from it.

- Maybe this extension could be a IDelayedInitializeService, like PodcastService ? This would prevent any increase in the visible startup time of banshee.

- I'm not sure about OnServiceStarted : it adds an event handler each time it's called, and I think OnSourceAdded ends up being called several times for each SourceAdded event.

Created attachment 135077 [details] [review]
Library Watcher extension for git repository (2)

- Removed the pidb file
- Added tabs where missed
- lines added in the proper place inside the system files
- Changed "Library Watcher Plugin" in "LibraryWatcherService"
- Removed the "using Banshee;" lines
- Removed "Log.Debug ..." and try/catch
- Implemented the Library Watcher as a delayed service

Ready for round 3 :P

Good to see the patch being updated, but isn't "Lybrary" still a typo?

(In reply to comment #60)
> "Lybrary Watcher Plugin" => typo

(In reply to comment #62)
> - The "Lybrary Watcher Plugin" typo is still there

(In reply to comment #64)
> Good to see the patch being updated, but isn't "Lybrary" still a typo?


(/me puts i18n hat on)

Guys, instead of repeatedly stating that there is a misspelling it would probably make much more sense to point out the correct spelling instead of repeating the same over and over again ("it's still wrong!"). Please keep in mind many contributors do not speak English as their first language, which implies that obvious mistakes may not be obvious at all for some of us. It can be quite demotivating to only point out mistakes people do not immediately see themselves; it's much more productive to be constructive and tell them how the mistake should be corrected. That said...

martellc: it should read "Library", not "Lybrary". Thanks for your work!

Created attachment 135161 [details] [review]
Library Watcher extension for git repository (3)

I'm sorry , I thought I Corrected the typo, but I did it again... :(
The last patch really fix it.

Created attachment 135828 [details]
Patch against last patch fixing style issues

Created attachment 135829 [details] [review]
Updated patch with style fixes

There are some other changes that need to happen before I'll be happy with this solution:

1) Now that each LibrarySource has its own BaseDirectory, we should create a watcher for each.  This also means we don't have to know or care if there is a music library etc - we can just watch all LibrarySources (which will include Podcasts).  We can still have one watch_thread, but we'll have multiple FileSystemWatchers.

2) I'm not keen on the

SearchForDeletedSongs ();
SearchForNewSongs ();

that are triggered every time there is a file rename, and on startup.  This is a very expensive operation.  And, if we're going to do it at all, we should reuse or merge this with the Rescan functionality.  And, for renames, can't we avoid this rescan entirely?  Keep in mind our 'Update file and folder names' option; it can trigger many many renames.

3) Instead of polling every 400ms to see if there are changes, we should use a ManualResetEvent or similar to sleep until there is something to do.

4) I believe calling "import_manager.KeepUserJobHidden = true;" will prevent the UserJob for regular importing from appearing too, which obviously isn't ok.

Created attachment 135891 [details] [review]
Updated patch, makes one SourceWatcher per LibrarySource, commented out the SearchFor* methods, for now, still needs a lot of work

this is the only thing holding me from switching completely from Rhythmbox.

Created attachment 138564 [details] [review]
Simplify some things and add manualResetEvent

I have change the system to a ManualResetEvent and some work to simplify code and avoid full reload!

Here's a problem I found while testing the latest patch :
After adding or deleting a file in my library folder (with cp or rm), banshee started using about 50% CPU until I quit it. Here the relevant part of my log output :

[Debug 18:49:16.447] Delayed Initializating Banshee.LibraryWatcher.LibraryWatcherService
relative = Music/Library/
[Debug 18:49:16.453] Started LibraryWatcher for Music (/home/lorentz/Music/Library/)
relative = Videos/
[Debug 18:49:16.454] Started LibraryWatcher for Videos (/home/lorentz/Videos/)
relative = Music/Podcasts/
[Debug 18:49:16.455] Started LibraryWatcher for Podcasts (/home/lorentz/Music/Podcasts/)
[Debug 18:49:16.465] Delayed Initializating Banshee.MediaEngine.PlayerEngineService
[Debug 18:49:16.511] (libbanshee:player) Using system (gst-plugins-good) equalizer element
[Debug 18:49:16.621] Player state change: NotReady -> Ready
[Debug 18:49:16.636] Enabling equalizer preset: New Preset
[Debug 18:49:16.642] Player state change: Ready -> Idle
[Debug 18:49:16.644] (libbanshee:player) Enabled ReplayGain
[Debug 18:49:16.660] (libbanshee:player) scaled volume: 1.000000 (ReplayGain) * 0.790000 (User) = 0.790000
[Debug 18:49:17.376] Starting - Downloading Cover Art
[Debug 18:49:17.378] Finished - Downloading Cover Art
[Debug 18:49:24.634] Starting - Saving Metadata to File
[Debug 18:49:24.640] Finished - Saving Metadata to File
OnDeleted /home/lorentz/Music/Library/06  Madonna - She's Not Me.mp3
[Debug 18:49:54.144] Starting - Downloading Cover Art
[Debug 18:49:54.146] Finished - Downloading Cover Art
[Debug 18:49:54.151] Starting - Saving Metadata to File
[Debug 18:49:54.157] Finished - Saving Metadata to File
[Debug 18:49:54.886] Starting - Downloading Cover Art
[Debug 18:49:54.896] Finished - Downloading Cover Art
[Debug 18:49:54.902] Starting - Saving Metadata to File
[Debug 18:49:54.908] Finished - Saving Metadata to File
[Debug 18:49:55.670] Starting - Downloading Cover Art
[Debug 18:49:55.671] Finished - Downloading Cover Art
[Debug 18:49:55.676] Starting - Saving Metadata to File
[Debug 18:49:55.681] Finished - Saving Metadata to File
[Debug 18:49:56.498] Starting - Downloading Cover Art
[Debug 18:49:56.505] Finished - Downloading Cover Art
[Debug 18:49:56.510] Starting - Saving Metadata to File
[Debug 18:49:56.516] Finished - Saving Metadata to File

The last 4 lines are repeated several times per second until I quit banshee.
I guess banshee somehow gets stuck in a loop.

Created attachment 138742 [details] [review]
fix the bug of infinite loop,  library which do not exist and improove hql for remove

fix the bug of infinite loop,  library which do not exist and improove hql for remove

Created attachment 143267 [details] [review]
fixes

Still work in progress.

Apparently a major bug in Mono breaks the watcher in some scenarios, see bnc#322330 [1]. In particular, moving files within the library and renaming folders don't work at all.

I will try to find a workaround...

[1] https://bugzilla.novell.com/show_bug.cgi?id=322330

I read your patch and think to 2 things to improove it:

- the system to remove special char from query to DB must be centralize in an util function as DbEscape (string str) {...}. (Maybe there is ever a such function somewhere?)

- Why add a timer of 2 sec? file have handle on it?
if yes, I am not sure it will be ok for a big file. Maybe there is a solution by trying action and if not work requeue...

else good patch (readonly on param OK / remove 3 List and add queue is great)

(In reply to comment #76)
> - the system to remove special char from query to DB must be centralize in an
> util function as DbEscape (string str) {...}. (Maybe there is ever a such
> function somewhere?)

Good idea. StringQueryValue is doing the same escaping, I will move to StringUtil.


> - Why add a timer of 2 sec? file have handle on it?
> if yes, I am not sure it will be ok for a big file. Maybe there is a solution
> by trying action and if not work requeue...

No, it's not because of locking. The events are actually triggered after successful file operations.

When the automatic file rename option is set and the track metadata is changed, the SaveTrackMetadataService first renames the file and then updates the database record with the new Uri. The queue delay ensures there's no conflict between the watcher and the metadata service.

I also checked the source code of Beagle and they use a delay to improve the tracking performance. It's only 50ms in their case, but it doesn't really matter as queue processing and event handling are in different threads.

Finally, the FileSystemWatcher never triggers OnRename event if a file/folder was moved to another folder. It's by design, if you check the RenamedEventArgs source code, you will see that it has a base path, and two file names. I'm currently not combining delete+create events as a single rename event, but I plan to, and the delay allows us to do it.

On a side note, I nearly finished fixing bnc#322330, I will keep you all updated on the progress.

Created attachment 143479 [details] [review]
next iteration

I managed to fix bnc#322330 [1], at least in my unscientific tests.

Now I'm not sure what to do with the fix. Even if it works well and gets committed soon, it will take at least a few months before it's in the next stable version of Mono, and several months before we can make it the minimum required version.

In this patch I bundled the patched version of InotifyWatcher and its dependencies and use is instead of the Mono version (but only under Linux). I know it's an ugly hack, but it works and allows to further test the extension. I'm open for suggestions on how to handle this elegantly.

That aside, the watcher works for the following scenarios:

 * Adding, deleting, renaming, and moving files and folders within the watched libraries will update the database. When moving/renaming the metadata such as ratings and scores should be preserved.
 * If the 'update file and folder names' option is on, the names are updated back to the defined schema.
 * Changing metadata using an external editor updates the database.
 * Moving tracks out of the watched folders will delete them from the database.

I'm 100% sure I missed some ways in which the library folder can be abused, so I encourage everyone to give the patch a try and report back.

Be sure to backup your media files and the database before you do!

[1] https://bugzilla.novell.com/show_bug.cgi?id=322330

Created attachment 144873 [details] [review]
Updated to git master

Just a quick note that the fix to bnc#322330 has been committed to the trunk and to the 2.4.x and 2.6.x branches. The bundled version of the InotifyWatcher can be removed as soon as Banshee has Mono 2.4.3+ as a dependency.

Bulk changing the assignee to banshee-maint@gnome.bugs to make it easier for people to get updated on all banshee bugs by following that address.  It's usually quite apparent who is working on a given bug by the comments and/or patches attached.

Review of attachment 144873 [details] [review]:

I pushed the patch to the 'watcher' branch.

Review of attachment 144873 [details] [review]:

I pushed the patch to the 'watcher' branch.

Review of attachment 144873 [details] [review]:

I pushed the patch to the 'watcher' branch.

Review of attachment 144873 [details] [review]:

I pushed the patch to the 'watcher' branch.

I've merged the watcher branch into git master.  I disabled the extension by default until we get some wider testing and polish.  Thanks so much Alexander, everybody who helped!