p.s.: 
there is one event where shotwell created *shotwell.jpg fotos for each raw+jpeg pair and shows each foto two times in the event: one time for the pair,one time for the shotwell development. 

In case of the other events, shotwell shows one image for raw+jpeg pair and it's own development. 

I hope we can somehow make sense of this behaviour which seems so far erratic to me :-(

as for 2) Please attach gdb to the running process and provide a backtrace of all running threads with t a a bt

Created attachment 335812 [details]
shotwell_gdb

Hi, 
thanks for the reply. 
I put the output into a text file hoping that is does belong there? 
Is this what you asked for? 
cheers

p.s. I am not exactly a gdb expert. If this is not what you need, I am happy to provide more information. ... as good as I can ;-)

two interesting things... Webkit seems to be running and it's regenerating the thumbnail cache in main thread...

cool. sounds like a starting point. anything else I can do?

Did you export photos with this run or was that a fresh start?

I merely started shotwell. 

I can start thinking about import after an hour or so, when shotwell starts being usable ;-)

That's weird then. can you run shotwell with export SHOTWELL_LOG=1 from console and attach shotwell.log from ~/.cache/shotwell ? thanks

Hi, 

here come the first lines of the log file: I get an error 500 if I try to upload an attachment. 

thanks

there are ~1500 more lines I guess for each image. Do you want an update after cpu usage has gone done? 


of this. L 5851 2016-09-25 17:51:05 [MSG] main.vala:385: Shotwell Fotoverwaltung 0.22.0
L 5851 2016-09-25 17:51:05 [DBG] Plugins.vala:316: Searching /home/philipp/.gnome2/shotwell/plugins for plugins ...
L 5851 2016-09-25 17:51:05 [DBG] Plugins.vala:135: Unable to search directory /home/philipp/.gnome2/shotwell/plugins for plugins: Error opening directory '/home/philipp/.gnome2/shotwell/plugins': No such file or directory
L 5851 2016-09-25 17:51:05 [DBG] Plugins.vala:316: Searching /usr/lib/shotwell/plugins for plugins ...
L 5851 2016-09-25 17:51:05 [DBG] Plugins.vala:316: Searching /usr/lib/shotwell/plugins/builtin for plugins ...
L 5851 2016-09-25 17:51:05 [DBG] Plugins.vala:424: Loaded SPIT module "Kerndaten Importdienst 0.22.0" (org.yorba.shotwell.data_imports.core_services) [/usr/lib/shotwell/plugins/builtin/shotwell-data-imports.so]
L 5851 2016-09-25 17:51:05 [DBG] Plugins.vala:424: Loaded SPIT module "Zusätzliche Shotwell-Veröffentlichungsdienste 0.22.0" (org.yorba.shotwell.publishing.extras) [/usr/lib/shotwell/plugins/builtin/shotwell-publishing-extras.so]
L 5851 2016-09-25 17:51:05 [DBG] Plugins.vala:424: Loaded SPIT module "Standard-Veröffentlichungsdienste 0.22.0" (org.yorba.shotwell.publishing.core_services) [/usr/lib/shotwell/plugins/builtin/shotwell-publishing.so]
L 5851 2016-09-25 17:51:05 [DBG] Plugins.vala:424: Loaded SPIT module "Standard-Diaschauübergänge 0.22.0" (org.yorba.shotwell.transitions) [/usr/lib/shotwell/plugins/builtin/shotwell-transitions.so]
L 5851 2016-09-25 17:51:05 [MSG] main.vala:43: Verifying database ...
L 5851 2016-09-25 17:51:05 [DBG] Db.vala:40: Database schema version 20 created by app version 0.15.0
L 5851 2016-09-25 17:51:06 [DBG] util.vala:171: next step: LibraryPhoto.init (0/27676)
L 5851 2016-09-25 17:51:08 [DBG] util.vala:171: next step: Video.init (27123/27676)
L 5851 2016-09-25 17:51:08 [DBG] util.vala:171: next step: Upgrades.execute (27152/27676)
L 5851 2016-09-25 17:51:08 [DBG] Upgrades.vala:85: Could not delete mimics: /home/philipp/.local/share/shotwell/mimics is not a directory
L 5851 2016-09-25 17:51:08 [DBG] util.vala:171: next step: Event.init (27152/27676)
L 5851 2016-09-25 17:51:09 [DBG] util.vala:171: next step: Tag.init (27617/27676)
L 5851 2016-09-25 17:51:09 [DBG] util.vala:171: next step: LibraryWindow (27672/27676)
L 5851 2016-09-25 17:51:10 [DBG] ConfigurationInterfaces.vala:331: ConfigurationFacade: engine reports property 'DISPLAY_BASIC_PROPERTIES' changed.
L 5851 2016-09-25 17:51:10 [DBG] ConfigurationInterfaces.vala:331: ConfigurationFacade: engine reports property 'DISPLAY_SEARCH_BAR' changed.
L 5851 2016-09-25 17:51:10 [DBG] ConfigurationInterfaces.vala:331: ConfigurationFacade: engine reports property 'EVENTS_SORT_ASCENDING' changed.
L 5851 2016-09-25 17:51:10 [DBG] LibraryWindow.vala:259: on_library_monitor_installed: /home/philipp/Bilder
L 5851 2016-09-25 17:51:10 [DBG] util.vala:171: next step: done (27672/27676)
L 5851 2016-09-25 17:51:10 [DBG] ConfigurationInterfaces.vala:331: ConfigurationFacade: engine reports property 'DISPLAY_SIDEBAR' changed.
L 5851 2016-09-25 17:51:10 [DBG] ConfigurationInterfaces.vala:331: ConfigurationFacade: engine reports property 'SHOW_WELCOME_DIALOG' changed.
L 5851 2016-09-25 17:51:10 [DBG] main.vala:195: 5.559964 seconds to Gtk.main()
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._Orangenlikör_7-06.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._Kardamon-schoko_8b-03.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._Kardamon Schoko-Stern_8-05.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._KaVaMuZi_7-07.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._Erdbeer-Stevia_7-03.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._Erdbee-Sahne_8b-04.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._Brombeer+Horizontal_8-01.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._Apfel-Zimt_8-04.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._.DS_Store
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/.DS_Store
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_6_kirsch-01.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_6_holunder-01.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_6_himbeer-01.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_6_haselnusshonig-01.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_2_mexicaner-19.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_2_kräuter.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._erdbeerlikör-02.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._eierlikör-28-28.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._Schokolikör normal_8b-05.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._ROSA_RUGOSA.pdf
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._voransicht_neu.png
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._schlehen_A4-01.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._sanddorn-02-02.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_mirabellenlikör-01_NEU_02-09-2013.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_ingwer.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_8b_schoko_vegan-07.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_6_zitrone-01.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_6_wallnuss-01.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_6_waldmeister-01.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/etiketten_alle/._etiketten_6_salbei-01.jpg
L 5851 2016-09-25 17:51:11 [WRN] DirectoryMonitor.vala:916: Skipping hidden file/directory /home/philipp/Bilder/NewYork/.dropbox
L 5851 2016-09-25 17:51:12 [DBG] ThumbnailCache.vala:213: import from source: [8955] /media/philipp/Pinguin/Bilder/Pralinen/IMG_0219.CR2
L 5851 2016-09-25 17:51:13 [DBG] ThumbnailCache.vala:213: import from source: [8956] /media/philipp/Pinguin/Bilder/Pralinen/IMG_0221.CR2
L 5851 2016-09-25 17:51:13 [DBG] ThumbnailCache.vala:213: import from source: [8957] /media/philipp/Pinguin/Bilder/Pralinen/IMG_0220.CR2
L 5851 2016-09-25 17:51:13 [DBG] ThumbnailCache.vala:213: import from source: [8958] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7934.CR2
L 5851 2016-09-25 17:51:14 [DBG] ThumbnailCache.vala:213: import from source: [8959] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7944.CR2
L 5851 2016-09-25 17:51:14 [DBG] ThumbnailCache.vala:213: import from source: [8960] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7933.CR2
L 5851 2016-09-25 17:51:15 [DBG] ThumbnailCache.vala:213: import from source: [8961] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7937.CR2
L 5851 2016-09-25 17:51:15 [DBG] ThumbnailCache.vala:213: import from source: [8962] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7947.CR2
L 5851 2016-09-25 17:51:16 [DBG] ThumbnailCache.vala:213: import from source: [8963] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7940.CR2
L 5851 2016-09-25 17:51:16 [DBG] ThumbnailCache.vala:213: import from source: [8964] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7943.CR2
L 5851 2016-09-25 17:51:16 [DBG] ThumbnailCache.vala:213: import from source: [8965] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7949.CR2
L 5851 2016-09-25 17:51:17 [DBG] ThumbnailCache.vala:213: import from source: [8966] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7953.CR2
L 5851 2016-09-25 17:51:17 [DBG] ThumbnailCache.vala:213: import from source: [8967] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7958.CR2
L 5851 2016-09-25 17:51:17 [DBG] ThumbnailCache.vala:213: import from source: [8968] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7964.CR2
L 5851 2016-09-25 17:51:18 [DBG] ThumbnailCache.vala:213: import from source: [8969] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7959.CR2
L 5851 2016-09-25 17:51:18 [DBG] ThumbnailCache.vala:213: import from source: [8970] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7961.CR2
L 5851 2016-09-25 17:51:18 [DBG] ThumbnailCache.vala:213: import from source: [8971] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7979.CR2
L 5851 2016-09-25 17:51:19 [DBG] ThumbnailCache.vala:213: import from source: [8972] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7968.CR2
L 5851 2016-09-25 17:51:19 [DBG] ThumbnailCache.vala:213: import from source: [8974] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7972.CR2
L 5851 2016-09-25 17:51:20 [DBG] ThumbnailCache.vala:213: import from source: [8973] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7980.CR2
L 5851 2016-09-25 17:51:20 [DBG] ThumbnailCache.vala:213: import from source: [8975] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7974.CR2
L 5851 2016-09-25 17:51:21 [DBG] ThumbnailCache.vala:213: import from source: [8976] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7975.CR2
L 5851 2016-09-25 17:51:21 [DBG] ThumbnailCache.vala:213: import from source: [8977] /media/philipp/Pinguin/Bilder/Kathrin30/IMG_7978.CR2
L 5851 2016-09-25 17:51:22 [DBG] ThumbnailCache.vala:213: import from source: [8978] /media/philipp/Pinguin/Bilder/BewerbungPortrait/IMG_7263.CR2
L 5851 2016-09-25 17:51:22 [DBG] ThumbnailCache.vala:213: import from source: [8979] /media/philipp/Pinguin/Bilder/BewerbungPortrait/IMG_7264.CR2
L 5851 2016-09-25 17:51:23 [DBG] ThumbnailCache.vala:213: import from source: [8980] /media/philipp/Pinguin/Bilder/BewerbungPortrait/IMG_7262.CR2
L 5851 2016-09-25 17:51:24 [DBG] ThumbnailCache.vala:213: import from source: [8981] /media/philipp/Pinguin/Bilder/BewerbungPortrait/IMG_7266.CR2
L 5851 2016-09-25 17:51:24 [DBG] ThumbnailCache.vala:213: import from source: [8982] /media/philipp/Pinguin/Bilder/BewerbungPortrait/IMG_7275.CR2

No, that's pretty clear it's the thumbnail cache. Is it always the same files on each startup?

well that's hard to say. 

I can tell you now, that ~8150 CR2 files are imported and that I have 10468 CR2 files in my photos-folder. 
(according to: find . -type f | sed -n 's/..*\.//p' | sort | uniq -c )

I will start shotwell again tomorrow and tell you the diff of the log files. 

thanks

I compared two shotwell logs: they seem to be identical, i.e. it seems to be the same files which are re-imported.

Created attachment 336456 [details]
shotwell log after startup

I think, the two above mentioned issues (auto-import and high cpu usage) are related after all: 

If you look at the attached shotwell.log, you see that from the last folder only one image is imported: 
IMG_2721.CR2

this image is also the only one in the folder which happens to have a shotwell-created sidekick: 

IMG_2721_CR2_shotwell.jpg

may be the "solution" to my problem is to let shotwell re-import the images *without* auto-development (i.e. without creating those *shotwell* files). 

But how?

How is the external drive mounted? Using gvfs or fstab?
If fstab, can you provide the line?

Which filesystem is it?

It is a ext4 filesystem. 
It is not mounted via fstab, I just wait for it to pop up if I plug it in.

I might have a branch to test for you later today that should take _a little bit_ pressure off the CPU usage...

https://git.gnome.org/browse/shotwell/log/?h=wip/faster-import might be of some help

Hi, 

I installed shotwell 0.25 from ppa. 
Unfortunately CPU usage (and the associated un-usability of shotwell) seem to remain. 

Is there anything else I can do? 

Even some pointers on how to leave shotwell (with my raw+jpg tags) would be greatly appreciated ;-) 


thanks a lot

Expected as there were no changes in that area.

Hi, 
Is there any progress / hope around? 

I currently have a photo collection in shotwell, which I can neither use (2h startup time) nor escape with (tags not in RAW+JPEG pairs). 

This situation is really really disappointing for me. 

If there is anything I can do 
- bounties
- research
- changing things on my side

My normal approach would be to try and re-import. However, this seems to have started the whole mess. In this case with shotwell re-developing raw images. 

Pretty much the only "solution" :-( I see right now is to do shotwells job: steal the raws from the library, deal with the whole issue of book keeping myself and hope that shotwell can deal with JPEG. 

thanks

Can you try the branch in comment 19?

I'm on my way ... managed to get a list of ~10 missing requirements ... and have to go to bed now ;-) 
Thanks, I'll let you know whether the comment 19 branch helps. Hopefully tomorrow.

I managed to check out and compile shotwell (configure does not check for valac).

The branch from comment 19 helped for the looking at pictures part. 

However, cpu ist still high and e.g. exporting during this high-cpu usage time is not possible. 

Is there anything else I can try?

Not yet, I still need to figure out why it thinks it has to rebuild the thumbnail cache. I might be able to provide some patch with extensive logging why it thinks it has to rebuild the cache.

cool. looking forward to applying that patch. 

Would you like that patch to be applied to standard-shotwell or to the branch you provided?

Ok, I noticed I have the same issue with two photos here. The issue is caused by you copying the library to a different place and shotwell being to stupid to notice that properly.

It's looking for the developments of the images in the old place, cannot see them and thinks it has to recreate them. 

It will then recreate them in the new library place and fails to properly update the database, so it will re-do the regenration all over again on every restart.

Did you already try to remove shotwell's database and have it recreate it?

I think this is the easiest way out here for you, though that is clearly a heavy bug on shotwell's side :-/

BIG Thank you from my side. !!!

How do I remove the database? 

Can I remove the database without loosing
- event-grouping
- rating (stars) 
- tags?

ah. No, of course not, didn't think about that. Let me think about something better.

As an intermediate step, can you move your ~/.local/share/shotwell/photo.db to a safe place and see re-creating the database fixes things - if you don't mind, of course

I moved the database from ~/.local/share/shotwell/data/photo.db 
so far shotwell is not doing anything, but I vaguely remember, that it takes some patience before the database is re-built.

hmm,
nothing happens. I guess, I have to use "import from folder"?

Ok, the database is re-built. 
Shotwell is useable again. Thanks! 

However, 
- grouping in events is lost.
- rating is lost for raw+jpeg pairs
- tags are lost for raw+jpeg pairs

In theory there is the option to write tags and rating into the files as opposed to keeping them in the database. However, this is another shotwell bug: writing tags and rating into files does not work for raw+jpeg pairs. If it was not for that last bug, I would consider this a usable solution: only having to do the grouping in events by hand.

and if I remove the *shotwell* file, which was created by shotwell and is not needed, shotwell re-imported the raw and jpeg pair ... as two individual files. ...

I'm still looking for finding a way to to fix your old database.

I think for the re-import as separate images is already a ticket open.

There is something else going on. When I move the library, I only see the behavior once.

Can you provide me your old (broken) database via private mail? Thanks.

So, I _think_ you can save your tags etc. by doing this on a copy of your original photo.db:

 - Stop shotwell
 - Go to your library, run this to remove all generated files: 
    find -name "*_CR2_*.jpg" -exec rm {} \;
 - Go to ~/.local/share/shotwell/data
    sqlite3 photo.db
     delete from BackingPhotoTable where filepath like '%_CR2_%.jpg';
     delete from PhotoTable where filename like '%_CR2_%.jpg';
     update PhotoTable set developer="EMBEDDED", develop_shotwell_id = -1, develop_camera_id = -1, develop_embedded_id = -1 where filename like '%.CR2';

Then re-start shotwell. The backing files will then be re-created on demand when opening the photo.

Hi, 

as far as I understand, this solution will remove the shotwell-generated files from my old database. 

But the issue with re-importing remains?

Or do you rather want me to try and see whether that issue vanishes if those auto-generated files are gone from the old database?

thanks for your help
cheers

Yes, that will remove all the autogenerated files from the database. Re-import will happen, but only once (so I hope) and then everything should be ok again, retaining your tags and grouping.

hmmmm, 
that did not work soo well :-(

moving back my old library seems to have let to major confusion on shotwell's side: 
the thumbnails and the actual images do not fit any more. 

It is kind of like playing memory (is this a German game? Where you have to turn cards and remember the image on the turned side?) : You click a small image (which is more or less random) and see which actual image is underneath. 

However, after having looked at the actual image, it stays and the random thumbnail is gone. The next two thumbnails are also correct now. 

Oh wait: this is not true in all cases. The thumbnail seems to stay random for the JPEG images. The correction of thumbnails seems to only work for the raw+jpeg cases.  

In addition, for each raw+jpeg image, I looked at, shotwell generates a *_CR2_embedded.jpg file. 

It also does not say Raw+jpeg but only Raw and Developer: Kamera
Ahh, as I feared: as shotwell does not understand any more that these are pairs, only the raw is handled. E.g. if I delete it, the JPG of the pair remains. 

However, the Re-Creating issue is gone now. Shotwell is not running with high cpu and is responsive. At the cost of not getting correct thumbnails for jpegs and having to click through all my raw+jpeg pairs and letting shotwell extract an embedded version. 

Thanks, hoping for the best.

update: if i put a checkmark on /enable monitor for new images, shotwell imports the jpeg of the raw+jpeg pairs as individual images. :-(

update: 
after removing the database and re-doing the above (comment 37) procedure the already re-created thumbnails are still there. I guess there is some shotwell-cache? 
Does shotwell re-create it if I delete it?

Damn. I don't understand the raw handling.

The thumbnails are cached in ~/.cache/shotwell/thumbs

Hi, 
I am sorry, but I am still out of a working shotwell :-( 

I had to tell shotwell to monitor the image folder. 
As mentioned above, this let to shotwell re-importing (re-developing raws to obtain Img*embedded.jpg). 

Now, all the raw+jpeg pairs are two times in shotwell. And actually three times on disc (raw+jpeg+embedded). Being multiple times on disc just steals space but cluttering my shotwell is a real mess. 

Any help is greatly appreciated. 

cheers

sorry, can you sum up the steps that you've done so far?

Ok , you're right :-)

moved the library to different location on external hard drive 
 -> led to shotwell using 100% CPU each startup for 1-2 hours
 => it turned out this was due to shotwell re-importing thousands of raw+jpeg pairs at startup
 
applied the commands from comment 37 to my database
 -> shotwell starts up normally
 -> shotwell lost the "pairing information" of raw+jpeg pairs
 -> shotwell got confused with all thumbnails 
 => turned all the jpeg (+undo) and improved (+undo) each event for raw+jpeg in order to get thumbnails back

turned on the "monitor image folder" option
 -> shotwell extracted an image from the raw (Img*embedded.jpg).
 
 => each raw+jpeg image is now twice in my shotwell collection.

- Is the continuous scanning now gone or is it still happening?
- I forgot about the thumbnails. what's even worse, tags are bound to thumbnail names (no clue why)
- I tried the moving and comment 37 here locally and it kind-of worked.

Can you strace -f shotwell for a while while scanning and attach the log?

the scanning stopped with commands from comment 37

it looks as if my tags are intact (well they are tied to the raw instead of the raw+jpeg)

do you need an strace although there does not seem to be any scanning / unusual startup?
thanks

No, strace would only have made sense if it kept scanning.

Hello, the issue as reported originally persists with Shotwell versions:
- 0.26.4
- 0.27.4

Applying solution from Comment 37 resolves the issue.

In my case photo library is located on NFS share, Shotwell database - locally. Library has .NEF files and Shotwell would start generating *_NEF_shotwell.jpg on startup despite the setting RAW Developer = Camera. I believe, this might have started after renaming NFS share.

Is there any specific info I can be helpful with to help to address this issue? Thanks!

So the behavior went away after you modified the shotwell db?

That's correct, after modifying Shotwell photo.db - 0.26.4 stopped generating _shotwell and _embedded jpgs.

(I didn't confirm this for 0.27.4)

I can confirm this for 0.27.4, modification of photo.db stopped generating _shotwell and _embedded jpgs.

And I can confirm that the thumbnails really mess things up badly with tags. Tags should not be bound to thumbnail nails IMHO.

They are not bound to thumbnails, the naming scheme is just massively unfortunate and needs fixing

The database rework will be part (or rather foundations) of 0.30 feature work

I switched from my self-compiled 0.27.4 version to the flatpack 0.30.1 release and rebuilt my database by deleting the old one. Many duplicate cr2 files were generated, so I used Comment 37 again to stop shotwell from creating more jpgs. Were you not able to fix this with the database rework you talked about in Comment 56?

Since the switch to 0.30.1, all my RAW photos show all kinds of tags that are not associated. I checked with exiftool and the tags do not match with the tags shotwell displays. So the tags saved with the RAW images are still correct. How can I fix this? I already tried rebuilding the database by deleting it, but the same thing happened again. Is this a separate bug?

Unfortunately, with the flatpak that's kind of expected since you start with a fresh database.

The rework did not happen.

-- GitLab Migration Automatic Message --

This bug has been migrated to GNOME's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/shotwell/-/issues/4758.