GNOME Bugzilla – Bug 722042
"Hide photos already imported" doesn't take account of "Rename imported files to lowercase" option
Last modified: 2019-01-25 22:08:56 UTC
While importing 1403 photos from an SD card, when I returned to the computer a while later I found it had crashed with an "out of memory" error having only imported 1173 of them. I rebooted and restarted shotwell. Selecting the SD card again for import, when I select the "Hide photos already imported" checkbox it makes no difference to the number of photos being displayed. It looks like this is because the files on the SD card are shown as e.g. DSCF1247.JPG while the imported ones are e.g. dscf1247.jpg (I have the "Rename imported files to lowercase" option selected in the preferences). Assuming this is the cause, perhaps Shotwell's import should use a case-insensitive match (or similarly convert to lower case for the comparison) when this option is in use?
When you tried to import those photos, did you see thumbnails in Shotwell? I ask because Shotwell uses the photos thumbnails in order to find already-imported photos. However, in some cases, when thumbnails aren't available (esp. w/ RAW and video), we have to fall back on filenames and sizes, which is ticketed at bug #717847. If you are seeing thumbnails, then this is a separate bug.
No RAW files involved, but it looks like things are actually not as straightforward as I thought, and possibly not to do with the filenames after all. Selecting the "Last Import" node reports "Items: 1173 Photos" in the bottom left corner, while selecting Library displays "Items: 1178 Photos" (there had been 5 others I'd imported from another camera on a previous occassion). But selecting "Mass Storage Camera" I realised there are actually "1454 Photos 2 Videos" on the card, which changes to report "1403 Photos 2 Videos" when I click the Hide checkbox. In either case I see thumbnails for all of them (at least, I think so - with that many it's hard to tell, but I don't see any that are obviously missing). However, examining the first example of one which does get hidden, I see that it's named DSCF1248.JPG on the card and dscf1248.jpg in the Library and Last Import list. But there's no obvious difference between that one (which behaves as expected) and DSCF1247.JPG/dscf1247.jpg & DSCF1249.JPG/dscf1249.jpg which remain visible whatever the state of the checkbox. Same image dimensions, taken on the same camera, timestamps within a couple of minutes, and I see thumbnails for all three in both ~/.cache/shotwell/thumbs/thumbs128 and thumbs360 (thumb0000000000000006.jpg, thumb0000000000000007.jpg & thumb0000000000000008.jpg) So I've no idea why all three don't behave the same way.
In this situation, Shotwell is using an MD5 hash of the image's thumbnails to determine what's already been imported. It may be that the thumbnail being pulled off the camera is either not the same as the thumbnail in the library (note that this is the thumbnail embedded in the JPEG itself, not the thumbnail Shotwell creates in the .cache directory). Can you determine what version of libgphoto2 you're running? It's important you're running version 2.5 or better. Also, can you run Shotwell like this: $ SHOTWELL_LOG=1 shotwell Load the camera images, click "Hide already imported", and quit. Then attach to this ticket ~/.cache/shotwell/shotwell.log The reason is, we've seen in the past gphoto2 add a couple of extra bytes to the tail of the thumbnail buffer when it pulls it off the camera. (This ruins duplicate detection.) We work around it in the code, but I'm curious if something similar is happening, and the log file might indicate that.
Created attachment 266735 [details] requested log file
Re: libgphoto version, does this answer the question? andy@annette:~$ dpkg --get-selections | grep libgphoto libgphoto2-2:i386 install libgphoto2-6:i386 install libgphoto2-l10n install libgphoto2-port0:i386 install libgphoto2-port10:i386 install From the Ubuntu Software Centre details, that's versions libgphoto2-2 2.4.14-2 libgphoto2-6 2.5.2-0ubuntu5 libgphoto2-l10n 2.5.2-0ubuntu5 libgphoto2-port0 2.4.14-2 libgphoto2-port10 2.5.2-0ubuntu5
Is that still a problem?
Closing this bug report as no further information has been provided. Please feel free to reopen this bug report if you can provide the information that was asked for in a previous comment. Thanks!