GNOME Bugzilla – Bug 326653
Path canonicalisation causing issues when upgrading
Last modified: 2006-02-07 11:41:21 UTC
Please describe the problem: When you add new files into an existing directory, there's no way to import only the new files into rhythmbox, or at least, I haven't found it. When I import the directory, all files, which were already in Rhythmbox, now appear two times. Steps to reproduce: 1. Put 10 files in a directory and import them into rhythmbox 2. Put another 5 files into that directory and import that directory again Actual results: You get 25 entries now, since the first 10 files are imported twice Expected results: There should have only been 15 entries Does this happen every time? yes Other information: look at amaroK; this player perfectly syncs (even in background) directories.
No, there's something specific to your library about this. Do you have symlinks or hard links set up so there are multiple possible paths for each file? Which version of rhythmbox were the existing files imported with? Previous versions did not canonicalise filenames before writing them to the database, so it's possible you might have imported the files with non-canonical names previously. Can you try doing this while running 'rhythmbox -d 2>&1 | grep <path>', where <path> is a significant portion of the path to the directory you're importing?
All songs were imported by Rhythmbox 0.9.2. I've now tried the same. In the directory /mp3/int/Gary Puckett & The Union Gap, there was exactly one song. Now I've imported the very same directory again, and voila, that one song is now twice in the Rhythmbox database. Here's the log: [0x8b97910] [queue_stat_uri] rhythmdb.c:1657 (21:51:54): queueing stat for "file:///mnt/hdb/mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3" [0x8cc3a68] [action_thread_main] rhythmdb.c:1790 (21:51:56): executing RHYTHMDB_ACTION_STAT for "file:///mnt/hdb/mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3" [0x8b97910] [rhythmdb_process_stat_event] rhythmdb.c:1345 (21:52:11): not modified: file:///mnt/hdb/mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3 [0x9101850] [queue_stat_uri] rhythmdb.c:1657 (21:53:23): queueing stat for "file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3" [0x8cc3a68] [action_thread_main] rhythmdb.c:1790 (21:53:23): executing RHYTHMDB_ACTION_STAT for "file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3" [0x8b97910] [rhythmdb_process_stat_event] rhythmdb.c:1363 (21:53:23): queuing a RHYTHMDB_ACTION_LOAD: file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3 [0x8cc3a68] [action_thread_main] rhythmdb.c:1802 (21:53:23): executing RHYTHMDB_ACTION_LOAD for "file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3" [0x8cc3a68] [rb_metadata_load] rb-metadata-gst.c:577 (21:53:23): loading metadata for uri: file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3 [0x8cc3a68] [rb_metadata_gst_load_tag] rb-metadata-gst.c:390 (21:53:23): uri: file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3 tag: title [0x8cc3a68] [rb_metadata_gst_load_tag] rb-metadata-gst.c:390 (21:53:23): uri: file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3 tag: artist [0x8cc3a68] [rb_metadata_gst_load_tag] rb-metadata-gst.c:390 (21:53:23): uri: file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3 tag: track-number [0x8cc3a68] [rb_metadata_gst_load_tag] rb-metadata-gst.c:390 (21:53:23): uri: file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3 tag: layer [0x8cc3a68] [rb_metadata_gst_load_tag] rb-metadata-gst.c:390 (21:53:23): uri: file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3 tag: mode [0x8cc3a68] [rb_metadata_gst_load_tag] rb-metadata-gst.c:390 (21:53:23): uri: file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3 tag: emphasis [0x8cc3a68] [rb_metadata_gst_load_tag] rb-metadata-gst.c:390 (21:53:23): uri: file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3 tag: bitrate [0x8cc3a68] [rb_metadata_load] rb-metadata-gst.c:711 (21:53:23): successfully read metadata for file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3 [0x8b97910] [rhythmdb_monitor_uri_path] rhythmdb.c:1066 (21:53:24): monitoring: /mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap [0x8b97910] [rhythmdb_property_model_insert] rhythmdb-property-model.c:513 (21:53:24): adding "Gary Puckett & The Union Gap": refcount 2 [0x8b97910] [queue_stat_uri] rhythmdb.c:1657 (21:53:24): queueing stat for "file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3" [0x8cc3a68] [action_thread_main] rhythmdb.c:1790 (21:53:24): executing RHYTHMDB_ACTION_STAT for "file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3" [0x8b97910] [rhythmdb_process_stat_event] rhythmdb.c:1345 (21:53:24): not modified: file:///mp3/int/Gary%20Puckett%20%26%20the%20Union%20Gap/Lady%20Willpower.mp3
I think, I've found the bug: on my system, there's a symlink /mp3 -> /mnt/hdb/mp3 It seems, like the first time, the mountpoint was resolved, and all songs were imported as in /mnt/hdb/mp3, while on the new import, the symlink /mp3 was not resolved... However, would it be possible in future to use title and artist as "primary key" (maybe additionally an (MD5)hash over the file) to avoid importing the same song with the same artist and title (and maybe even with the same content - therefor the (MD5) hash) in the database?
I had the same problem. You might want to try using Nautilus' "Connect to server" to make a "bookmark" of your directory. It will show up in the places menu, and in file selectors.
RB now has code that "canonicalises" file names, so symlinks shouldn't cause duplicates any more. However, it can cause duplicates if you are upgrading from an earlier version and there are symlinked directories. The way to do this would be to have RB canonicalise all entry URIs on startup. This is slow because it requires checking whether each part of the URI is a symlink (and resolving it, if it is) for every entry - so we would only want to do it once. We really need to have some version information in the rhythmdb.xml file, which is incremented whenever we need to detect whether upgrade processing needs to happen.
Retitling to make problem clearer. What we really need to do is have some versioning information in rhythmdb.xml, so that we can do upgrade processing once.
I had weird problems with tracks appearing as they were new from time to time. I noticed the common aspect is that the paths to files all contained "&" as yours do above, and were added before Jan 17 when path canonicalisation changed, I filed bug #329988 on this. Could be related?
(In reply to comment #7) > as they were new Should have read: as *if* they were new
All path canonicalisation isses due to upgrading should be fixed by the patch on bug 329988, which has been committed. This shouldn't occur any more with cvs HEAD, but feel free to reopen if anyone can reproduce it with a rhythmdb.xml which is version 1.1