---- Reported by adam@yorba.org 2010-09-16 08:54:00 -0700 ----

Original Redmine bug id: 2566
Original URL: http://redmine.yorba.org/issues/2566
Searchable id: yorba-bug-2566
Original author: Adam Dingle
Original description:

From Joao Coelho <norphon@hotmail.com>:

On Ubuntu 10.04 tried to import 11,000 photos from F-Spot database. First
stage completed successfully. Crashed during second stage where photos are
flashed in sequence and events and tags are created in Shotwell. Memory use
keeps climbing until system runs out of memory and it all falls over.



---- Additional Comments From shotwell-maint@gnome.bugs 2013-05-01 11:40:00 -0700 ----

### History

####

#1

Updated by Jim Nelson about 3 years ago

A user on Launchpad reported something similar:
https://bugs.launchpad.net/bugs/588569

####

#2

Updated by Vera Yin about 3 years ago

I've done some tests on my system (Ubuntu 10.10 beta) and observed the
following:

a) importing an F-Spot library of 10,000 photos in 0.7.2 – memory use climbs
to 1GB halfway through and stays there

b) importing a folder of 10,000 photos in 0.7.2 – memory use climbs to 700MB
halfway through and stays there

c) importing a folder of 10,000 photos in trunk – memory use climbs to 1GB
40% of the way through, drops to 800MB by the end

####

#3

Updated by Jim Nelson about 3 years ago

Marked [](http://trac.yorba.org/search?q=ticket%3A2349)#2349 as a duplicate.

####

#4

Updated by Jim Nelson about 3 years ago

  * **Status** changed from _Open_ to _5_
  * **Resolution** set to _fixed_
  * **% Done** set to _100_

####

#5

Updated by Vera Yin about 3 years ago

Some updated numbers after Jim's fix -

Import of 10,000 photos (from disk, linking) took 18 minutes, memory usage
peaking at 160MB.

Import of 10,000 photos (from disk, copying) took 31 minutes, memory usage
peaking at 135MB.

Import of 157 photos (from camera) took 30 seconds, memory usage peaking at
80MB.

####

#6

Updated by Jim Nelson about 3 years ago

Fantastic. This is well within normal bounds.

####

#7

Updated by Mike - about 3 years ago

  * **Status** changed from _5_ to _4_
  * **Resolution** deleted (<strike>_fixed_</strike>)
  * **% Done** changed from _100_ to _0_
  * **Priority** deleted (<strike>_Urgent_</strike>)

I had a rather large collection to import (40K+ images), and it took the repo
Shotwells three imports to do them (the first two caused memory usage grow so
large as to crash the program AND my window compositor (!)).

The bad news: I'm still noticing the issue in trunk. I downloaded, configured,
and compiled this morning, then when I do a sizeable import, the window
becomes unresponsive, and my memory usage climbed to nearly a gigabyte
(989MiB). This was just importing around 1/20th of my library; I suspect an
attempted full import would cause the same issue I was seeing before. Moreover
once the import concluded, the memory used during import was **not** released.

How can I help you identify and fix this here? By the way, I was **very**
reluctant to reopen this, but I want to make sure we get this bug squashed;
this will be a hindrance to adoption of previous F-Spot users like myself.

####

#8

Updated by Mike - about 3 years ago

I can provide some more info. I just reproduced my issue:

Import (from disk, linking) of 5,544 images took approximately 20 minutes and
memory usage peaked (and remained) at 1.2GiB.

Shotwell 0.7.2+trunk

Ubuntu 10.04

Kernel 2.6.32.25

2.0 GiB memory

Athlon 64 processor 3200+

####

#9

Updated by Jim Nelson about 3 years ago

  * **Priority** set to _Urgent_

Hmm … definitely concerned about you seeing this running from trunk. One
thing first off: What version of gexiv2 are you using?

`$ pkg-config --modversion gexiv2`

We had a memory leak there as well a while back. Would be best if you were
using 0.2.1 (or better, trunk) to see if this problem persists.

Also, are you importing videos? If so, how many?

####

#10

Updated by Mike - about 3 years ago

I'm using gexiv2 0.2.1. That was 5544 images and 114 videos.

####

#11

Updated by Adam Dingle about 3 years ago

  * **Subject** changed from _large import from F-Spot crashes_ to _large import uses lots of memory, sometimes leading to crash_

Renaming ticket to indicate this this is not F-Spot specific and involves a
memory leak.

####

#12

Updated by Jim Nelson about 3 years ago

  * **Status** changed from _4_ to _5_
  * **Resolution** set to _fixed_
  * **% Done** changed from _0_ to _100_

r2371

####

#13

Updated by Adam Dingle about 3 years ago

  * **Status** changed from _5_ to _4_
  * **Resolution** deleted (<strike>_fixed_</strike>)
  * **% Done** changed from _100_ to _0_

I had to back outr2371and the associatedr2374because they caused a hang on
import; see#2864. So I'm reopening this ticket. Jim can look at this when he's
back at Yorba later this week.

####

#14

Updated by Jim Nelson almost 3 years ago

  * **Status** changed from _4_ to _5_
  * **Resolution** set to _fixed_
  * **% Done** changed from _0_ to _100_

Threads could become starved in certain cases (race condition). Also
discovered a problem with a non-thread-safe code being called by background
threads; this only occurs when files are copied into the library during
import.

This patch should solve the starvation and locking problems. r2387

####

#15

Updated by Charles Lindsay 7 months ago

  * **Status** changed from _5_ to _Fixed_



--- Bug imported by chaz@yorba.org 2013-11-25 21:47 UTC  ---

This bug was previously known as _bug_ 2566 at http://redmine.yorba.org/show_bug.cgi?id=2566

Unknown Component 
   Using default product and component set in Parameters 
Unknown version " in product shotwell. 
   Setting version to "!unspecified".
Unknown milestone "unknown in product shotwell. 
   Setting to default milestone for this product, "---".
Setting qa contact to the default for this product.
   This bug either had no qa contact or an invalid one.