After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 775196 - Duplicate detection broken with RAW files
Duplicate detection broken with RAW files
Status: RESOLVED FIXED
Product: shotwell
Classification: Other
Component: camera
unspecified
Other Linux
: Normal critical
: 0.26
Assigned To: Shotwell Maintainers
Shotwell Maintainers
Depends on:
Blocks:
 
 
Reported: 2016-11-27 22:17 UTC by Amos Brocco
Modified: 2017-01-10 17:57 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Raw duplicate detection (added exposure timestamp comparison) (2.06 KB, patch)
2016-11-27 22:17 UTC, Amos Brocco
committed Details | Review

Description Amos Brocco 2016-11-27 22:17:32 UTC
Created attachment 340866 [details] [review]
Raw duplicate detection (added exposure timestamp comparison)

Current duplicate detection for RAW files works by comparing the filename and file size. Unfortunately when importing RAW files from an sdcard I noticed that Shotwell wrongly assumed that many files were duplicated and already stored in the library. Upon further examination I discovered that the criteria used for duplicate detection with RAW files are based only on the basename and file size.

Now I have and use three cameras that are exactly the same make and model (Sony A55), and I frequently have pictures generated on these different cameras at different times that share the same filename (something like DSC0XXXXX.ARW) and the SAME SIZE.

This means that Shotwell will import the first copy without issues but will then detect files from other cameras as duplicates when there's a match (name + filesize). 

Now if the option to hide already imported files is set, Shotwell won't show me some of the images and will not import them. When I import hundreds of photos at once the fact that some of the photos are missing will get easily unnoticed (I typically don't check every single file, I just select to import all images). Unfortunately this way I might lose some pictures, thinking that all photos have been safely copied on my computer: in particular, if I format my sdcard all photos that were mistakenly considered as duplicates by Shotwell will be unrecoverably lost.

Note: I marked this bug as major because it can cause data losses to an end-user (as I've already experienced myself)

In attachment you will find a patch that should fixe this bug by comparing not only the filename and filesize but also the exposure timestamp.
Comment 1 Jens Georg 2016-11-28 09:44:10 UTC
Potential data loss in unacceptable. thanks for analyzing this.