GNOME Bugzilla – Bug 717532
RAW thumbnail generation is CPU intensive
Last modified: 2021-05-19 12:48:16 UTC
---- Reported by shotwell-maint@gnome.bugs 2011-05-07 22:25:00 -0700 ---- Original Redmine bug id: 3586 Original URL: http://redmine.yorba.org/issues/3586 Searchable id: yorba-bug-3586 Original author: Pedro Côrte-Real Original description: I decided to test shotwell by importing all my library into it. It consists of 300GB in 38k files of which 19k are RAW files. I currently manage it with f-spot and imported in place (without copying). The test was done on a Lenovo X61 with a 2.2GHz Core2Duo and shotwell 0.9.2 as shipped by ubuntu natty. Shotwell took around 15 hours to do the full import and that's without generating all the mimics. This seemed excessive so I did some tests on the same machine: Using the dcraw settings f-spot uses and piping that into convert: $time for i in `find ~/Photos/ -name "*.ARW" | sort -R | tail -n 100`; do dcraw -h -w -c -t 0 $i | convert -geometry 128×128 - $(basename $i).jpg ; done real 1m44.282s user 1m4.250s sys 0m10.490s So for 38k photos it would be 1.75m*380 = 11 hours, which sounds around the right ballpark for what we got. Then I tested importing 2GB of RAW photos into both f-spot and shotwell. The results were: Importing from CF card (so needing to copy) f-spot: ~3m10s with <50% of a core used shotwell: ~3m40s with 100% of a core used Importing from ~/Pictures (in place) f-spot: ~1m20s with 100% of a core used shotwell: ~4m07s with <100% of a core used (that this ended up slower is odd and probably just means my methodology was too sloppy) Clearly there's some improvement potential here. f-spot is doing the half-size RAW conversion trick (-h) but so is shotwell I think. I wonder if there's any technical limitation to doing raw conversions in quarter or eigth size to speed up conversions more. It could also help with not needing mimics any more. Related issues: related to shotwell - 3051: Camera timing out during import (Open) related to shotwell - 6118: Excessive memory use on changing developer (Open) related to shotwell - 1856: System sluggish during a drag-and-drop RAW import (Open) ---- Additional Comments From shotwell-maint@gnome.bugs 2013-05-16 14:44:00 -0700 ---- ### History #### #1 Updated by Pedro Côrte-Real over 2 years ago Note that one reason f-spot probably saturates the CPU a bit better is that they fork out for dcraw so they probably end up using the two cores, one for the raw conversions, another for everything else. They don't seem to be forking more than one conversion at a time though. This isn't enough to explain the difference though as there's more than a 2x difference between f-spot and shotwell. #### #2 Updated by Adam Dingle over 2 years ago * **Priority** set to _High_ Would be good to investigate this more. #### #3 Updated by Lucas Beeler about 2 years ago * **Description** updated (diff) * **Target version** set to _0.12_ Since we're going to be digging into the RAW developer stuff again in 0.12, we might be able to attack this. #### #4 Updated by Jim Nelson about 2 years ago * **Description** updated (diff) * **Target version** deleted (<strike>_0.12_</strike>) #### #5 Updated by Jim Nelson 11 months ago * **Target version** set to _0.14.0_ #### #6 Updated by Jim Nelson 11 months ago * **Category** set to _raw_ #### #7 Updated by Jim Nelson 11 months ago * **Tracker** changed from _Feature_ to _Bug_ * **Priority** changed from _High_ to _Normal_ We should retest this with latest Shotwell and LibRaw and see how it looks. #### #8 Updated by Jim Nelson 10 months ago * **Target version** changed from _0.14.0_ to _0.15.0_ #### #9 Updated by Jim Nelson 10 months ago Also, this may be a duplicate of #1856. #### #10 Updated by Jim Nelson 8 months ago * **Target version** changed from _0.15.0_ to _0.16.0_ #### #11 Updated by Jim Nelson 6 months ago * **Target version** deleted (<strike>_0.16.0_</strike>) --- Bug imported by chaz@yorba.org 2013-11-25 21:52 UTC --- This bug was previously known as _bug_ 3586 at http://redmine.yorba.org/show_bug.cgi?id=3586 Unknown version " in product shotwell. Setting version to "!unspecified". Unknown milestone "unknown in product shotwell. Setting to default milestone for this product, "---". Setting qa contact to the default for this product. This bug either had no qa contact or an invalid one. Resolution set on an open status. Dropping resolution
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/shotwell/-/issues/2675.