After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 717532 - RAW thumbnail generation is CPU intensive
RAW thumbnail generation is CPU intensive
Status: RESOLVED OBSOLETE
Product: shotwell
Classification: Other
Component: raw
unspecified
Other All
: Normal normal
: ---
Assigned To: Shotwell Maintainers
Shotwell Maintainers
Depends on:
Blocks:
 
 
Reported: 2011-05-08 05:25 UTC by Shotwell Maintainers
Modified: 2021-05-19 12:48 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Charles Lindsay 2013-11-25 21:52:35 UTC


---- Reported by shotwell-maint@gnome.bugs 2011-05-07 22:25:00 -0700 ----

Original Redmine bug id: 3586
Original URL: http://redmine.yorba.org/issues/3586
Searchable id: yorba-bug-3586
Original author: Pedro Côrte-Real
Original description:

I decided to test shotwell by importing all my library into it. It consists of
300GB in 38k files of which 19k are RAW files. I currently manage it with
f-spot and imported in place (without copying). The test was done on a Lenovo
X61 with a 2.2GHz Core2Duo and shotwell 0.9.2 as shipped by ubuntu natty.
Shotwell took around 15 hours to do the full import and that's without
generating all the mimics. This seemed excessive so I did some tests on the
same machine:

Using the dcraw settings f-spot uses and piping that into convert:

$time for i in `find ~/Photos/ -name "*.ARW" | sort -R | tail -n 100`; do

dcraw -h -w -c -t 0 $i | convert -geometry 128×128 - $(basename $i).jpg ;

done

real 1m44.282s

user 1m4.250s

sys 0m10.490s

So for 38k photos it would be 1.75m*380 = 11 hours, which sounds around the
right ballpark for what we got.

Then I tested importing 2GB of RAW photos into both f-spot and shotwell. The
results were:

Importing from CF card (so needing to copy)

f-spot: ~3m10s with <50% of a core used

shotwell: ~3m40s with 100% of a core used

Importing from ~/Pictures (in place)

f-spot: ~1m20s with 100% of a core used

shotwell: ~4m07s with <100% of a core used (that this ended up slower is odd
and probably just means my methodology was too sloppy)

Clearly there's some improvement potential here. f-spot is doing the half-size
RAW conversion trick (-h) but so is shotwell I think. I wonder if there's any
technical limitation to doing raw conversions in quarter or eigth size to
speed up conversions more. It could also help with not needing mimics any
more.

Related issues:
related to shotwell - 3051: Camera timing out during import (Open)
related to shotwell - 6118: Excessive memory use on changing developer (Open)
related to shotwell - 1856: System sluggish during a drag-and-drop RAW
import (Open)



---- Additional Comments From shotwell-maint@gnome.bugs 2013-05-16 14:44:00 -0700 ----

### History

####

#1

Updated by Pedro Côrte-Real over 2 years ago

Note that one reason f-spot probably saturates the CPU a bit better is that
they fork out for dcraw so they probably end up using the two cores, one for
the raw conversions, another for everything else. They don't seem to be
forking more than one conversion at a time though. This isn't enough to
explain the difference though as there's more than a 2x difference between
f-spot and shotwell.

####

#2

Updated by Adam Dingle over 2 years ago

  * **Priority** set to _High_

Would be good to investigate this more.

####

#3

Updated by Lucas Beeler about 2 years ago

  * **Description** updated (diff)
  * **Target version** set to _0.12_

Since we're going to be digging into the RAW developer stuff again in 0.12, we
might be able to attack this.


####

#4

Updated by Jim Nelson about 2 years ago

  * **Description** updated (diff)
  * **Target version** deleted (<strike>_0.12_</strike>)

####

#5

Updated by Jim Nelson 11 months ago

  * **Target version** set to _0.14.0_

####

#6

Updated by Jim Nelson 11 months ago

  * **Category** set to _raw_

####

#7

Updated by Jim Nelson 11 months ago

  * **Tracker** changed from _Feature_ to _Bug_
  * **Priority** changed from _High_ to _Normal_

We should retest this with latest Shotwell and LibRaw and see how it looks.

####

#8

Updated by Jim Nelson 10 months ago

  * **Target version** changed from _0.14.0_ to _0.15.0_

####

#9

Updated by Jim Nelson 10 months ago

Also, this may be a duplicate of #1856.

####

#10

Updated by Jim Nelson 8 months ago

  * **Target version** changed from _0.15.0_ to _0.16.0_

####

#11

Updated by Jim Nelson 6 months ago

  * **Target version** deleted (<strike>_0.16.0_</strike>)



--- Bug imported by chaz@yorba.org 2013-11-25 21:52 UTC  ---

This bug was previously known as _bug_ 3586 at http://redmine.yorba.org/show_bug.cgi?id=3586

Unknown version " in product shotwell. 
   Setting version to "!unspecified".
Unknown milestone "unknown in product shotwell. 
   Setting to default milestone for this product, "---".
Setting qa contact to the default for this product.
   This bug either had no qa contact or an invalid one.
Resolution set on an open status.
   Dropping resolution 

Comment 1 GNOME Infrastructure Team 2021-05-19 12:48:16 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to GNOME's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/shotwell/-/issues/2675.