After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 718616 - Be smarter about pre-caching images or don't cache altogether
Be smarter about pre-caching images or don't cache altogether
Status: RESOLVED FIXED
Product: shotwell
Classification: Other
Component: pipeline
unspecified
Other All
: High normal
: 0.20.1
Assigned To: Shotwell Maintainers
Shotwell Maintainers
Depends on:
Blocks:
 
 
Reported: 2012-05-04 06:06 UTC by clinton
Modified: 2014-09-26 21:19 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Charles Lindsay 2013-11-25 21:57:13 UTC


---- Reported by clinton@yorba.org 2012-05-04 11:06:00 -0700 ----

Original Redmine bug id: 5202
Original URL: http://redmine.yorba.org/issues/5202
Searchable id: yorba-bug-5202
Original author: Clinton Rogers
Original description:

Several users on the mailing list have complained about severe performance
regressions and/or crashes with very large images in direct-edit mode. As
larger images become more common, it's only going to make the problems with
the current scheme more evident.

Based on a conversation I had with Jim yesterday, the least painful way
forward may be to remove the 'unmodified' cache mechanism and go back to the
scaled decoding mechanism we had before (as I say this, I fully acknowledge
that doing so will add a lot of complexity to coordinate-sensitive editing
tools* that I originally wrote the current pipeline mechanism to remove or
sidestep).

_Tentatively_ marking for 0.13 (subject to triaging and priority change by the
leads).

*We may be able to make this less painful and/or tidier by adding some kind of projection matrix mechanism that maps image space to screen space.

Related issues:
related to shotwell - 4777: Very large images load in EoG but hang
Shotwell (Open)
related to shotwell - 4944: autoenhance on huge image exceeds available
memory and cr... (Open)
related to shotwell - 4937: [direct edit] moving to next/previous image
is slow (Open)



---- Additional Comments From shotwell-maint@gnome.bugs 2013-05-16 14:44:00 -0700 ----

### History

####

#1

Updated by Lucas Beeler over 1 year ago

  * **Subject** changed from _Suggestion: revert the pipeline to the 'old' behaviour._ to _Be smarter about pre-caching images or don't cache altogether_

We may be able to do the caching more intelligently. I have some ideas. Given
the time we spent on coordinate transformations to get straightening to work
in the 0.12 cycle I'd like to avoid opening up that rat's nest again if
possible.

####

#2

Updated by Laura Khalil over 1 year ago

I've noticed that moving through a library with photos that have a lot of
pixelation, there is severe lag when trying to move from one photo to another.
When I scroll through photos that have are less pixelated, I can right arrow
through them just fine. Clint suggested I add a comment as it's probably
related to this bug.

####

#3

Updated by Clinton Rogers over 1 year ago

For those following the state of this bug:

http://redmine.yorba.org/projects/shotwell/repository/revisions/master/annotat
e/src/Photo.vala#L3022

http://redmine.yorba.org/projects/shotwell/repository/revisions/master/annotat
e/src/Photo.vala#L3035 ⬅ problem areas

The first line tries to make a full-size copy of the image in memory so that
a) we don't have to load it from disk again every time it's asked for and b)
the pipeline (where all the rotate/straighten/crop/colour modification work
happens) becomes very simple (coordinate-sensitive tools can work in image
coordinates all the time without worrying about projecting to or from screen
coordinates, and they can freely modify image data without ruining anything,
since we have an unmodified copy cached at all times, etc).

This seemed like a good idea at the time (and preliminary testing with images
below a certain size revealed that the application actually got _faster_);
unfortunately, once we have an image file that expands to more than about 1 GB
of image data, it starts to strain a bit under the weight of a 1 GB image
_and_ a copy of said image (recall that it keeps a 'pristine' one in memory).
And of course, the larger the image, the more significant the memory usage
becomes, to the point where an allocation could fail image is too large.

####

#4

Updated by Adam Dingle about 1 year ago

  * **Priority** changed from _Normal_ to _High_

####

#5

Updated by Adam Dingle about 1 year ago

  * **Target version** deleted (<strike>_0.13_</strike>)

####

#6

Updated by Jim Nelson 11 months ago

  * **Target version** set to _0.14.0_

####

#7

Updated by Jim Nelson 11 months ago

  * **Category** set to _library-mode_

####

#8

Updated by Jim Nelson 11 months ago

  * **Category** changed from _library-mode_ to _pipeline_
  * **Target version** changed from _0.14.0_ to _0.15.0_

We'll try and get to this in 0.15.

####

#9

Updated by Jim Nelson 11 months ago

  * **Tracker** changed from _Task_ to _Bug_

####

#10

Updated by Jim Nelson 8 months ago

  * **Target version** changed from _0.15.0_ to _0.16.0_

####

#11

Updated by Jim Nelson 6 months ago

  * **Target version** deleted (<strike>_0.16.0_</strike>)



--- Bug imported by chaz@yorba.org 2013-11-25 21:57 UTC  ---

This bug was previously known as _bug_ 5202 at http://redmine.yorba.org/show_bug.cgi?id=5202

Unknown version " in product shotwell. 
   Setting version to "!unspecified".
Unknown milestone "unknown in product shotwell. 
   Setting to default milestone for this product, "---".
Setting qa contact to the default for this product.
   This bug either had no qa contact or an invalid one.
Resolution set on an open status.
   Dropping resolution 

Comment 1 Jim Nelson 2014-09-26 21:19:13 UTC
The untransformed full-sized cache was the root cause of a rather nasty bug: bug #715198.  I've revamped how the cache works to be more modest and reduce load.  Hence, I'm closing this ticket.