GNOME Bugzilla – Bug 718989
[interim] Rewrite video thumbnailer pipeline to take advantage of GStreamer 1.0 features & idioms
Last modified: 2015-01-02 19:19:42 UTC
---- Reported by lucas@yorba.org 2013-03-12 15:11:00 -0700 ---- Original Redmine bug id: 6520 Original URL: http://redmine.yorba.org/issues/6520 Searchable id: yorba-bug-6520 Original author: Lucas Beeler Original description: None ---- Additional Comments From shotwell-maint@gnome.bugs 2013-10-23 16:49:00 -0700 ---- ### History #### #1 Updated by Lucas Beeler 8 months ago * **Target version** changed from _0.14.0_ to _0.15.0_ #### #2 Updated by Clinton Rogers 8 months ago For at least [one user](http://lists.yorba.org/pipermail/shotwell/2013-March/004653.html) , not doing things the officially-blessed way may be leading to ruined thumbnails; it seems that we are, in some cases, thumbnailing an intermediary frame, but without having seen the keyframe before it. Long story short, this causes a grey pixbuf with 'ghostlike' images (the changes from the previous frame). #### #3 Updated by Lucas Beeler 8 months ago * **Subject** changed from _Rewrite video thumbnailer pipeline to take advantage of GStreamer 1.0 features & idioms_ to _[interim] Rewrite video thumbnailer pipeline to take advantage of GStreamer 1.0 features & idioms_ #### #4 Updated by Jim Nelson 6 months ago * **Target version** deleted (<strike>_0.15.0_</strike>) #### #5 Updated by Timo Jyrinki about 1 month ago * **File** shotwell-video-thumbnailer.vala added I ended up here eventually, but let me give the background. I started by trying out 0.14 Shotwell on Ubuntu 12.04 LTS, and it turned out video thumbnailing stopped working because the Yorba PPA only offers the base GStreamer 1.0 packages. I got the rest of the needed ones from https://launchpad.net/~gstreamer-developers/+archive/ppa, but then bumped into that all my current H.264 videos from camera give these grey frames as thumbnails. I tried reverting to Shotwell 0.12 but apparently the database format had changed and I couldn't get the library visible in 0.12 anymore. I tried out Shotwell 0.15 as well (after reverting the one commit that bumped GTK dependency to 3.6 - rest of the dependencies were in either Yorba or GStreamer PPA), but with the same results. So I then compared the result to a machine running Ubuntu 13.10, and found that the problem persists there with 0.15 from archives and everything updated. I tried to tweak around the GStreamer options in shotwell thumbnailer but didn't find a combination that would improve the situation. Finally, I then crafted a solution/workaround yesterday and now checked that it still seems to work. I switched out the thumbnailer to an attached one which basically just dumbly wraps around totem-thumbnailer. It's working wonders for me - all the videos get a nice enough thumbnail. I manually removed the erroneous ones from .cache/shotwell/* (useful to search for video-* and sort by size), and now I've only good thumbnails. I understand this is just a hack/workaround, but as mentioned I couldn't find a working combination otherwise. I think it's useful however for people who want to have a quick and dirty solution. The totem thumbnailer might give hints on how to do the same in Shotwell: https://git.gnome.org/browse/totem/plain/src/totem-video-thumbnailer.c I now have a side issue that I don't get the dates read correctly from the videos, I need to look at that - resolution is wrong, for example 360x169, and no other meta information is gotten. That results in that new videos are not easily found from the database since they're at the very bottom. In summary, hopefully the attached wrapper is of some use to someone struggling with the same problems. #### #6 Updated by Timo Jyrinki about 1 month ago Ok a small update to my final solution (workaround). Sorry for spamming without an actual code fix. 1. Using '--raw' instead of '-s 360' for totem-video-thumbnailer makes sense and fixes the (reported) resolution issue I mentioned. 2. I did not have a metadata reading regression, but encountered #3227 when I was moving the MTS files around to and from library. Reimporting them from a DCIM folder on another partition fixed the problem and the dates were ready properly again as well. -Timo, now a happy Shotwell 0.15 on Ubuntu 12.04 LTS user ps. I used https://launchpad.net/~vala-team/+archive/ppa for the Vala 0.22 backport #### #7 Updated by Jim Nelson 29 days ago * **Status** changed from _Open_ to _Review_ * **Target version** set to _0.16.0_ --- Bug imported by chaz@yorba.org 2013-11-25 21:59 UTC --- This bug was previously known as _bug_ 6520 at http://redmine.yorba.org/show_bug.cgi?id=6520 Imported an attachment (id=262611) Unknown milestone "unknown in product shotwell. Setting to default milestone for this product, "---". Setting qa contact to the default for this product. This bug either had no qa contact or an invalid one. Resolution set on an open status. Dropping resolution
Created attachment 270539 [details] [review] make shotwell thumbnailer a wrapper around totem thumbnailer
Created attachment 270540 [details] make shotwell thumbnailer a wrapper around totem thumbnailer Uh oh, the previous one went in marked as a patch. But this is just to have the actual version I'm using attached to the bug. (the --raw change). I'll continue to use this probably with Ubuntu 14.04, applying on top of Shotwell 0.17.1.
Timo, the problem with this approach is that it then makes Totem a hard dependency for Shotwell. We had a number of users complain about this because Totem itself has hard dependencies on a large number of libraries that users found undesirable to install. For some history on this, see bug #717138, bug #717696, and especially bug #717671. If the Totem video thumbnailer was broken out into a separate package, and that package had a reasonable dependency set, we could consider this approach. (In fact, I would suggest simply getting rid of shotwell-video-thumbnailer entirely and let the main process launch totem-video-thumbnailer.) Until then, we can't go this direction.
Hi, it's the Mr. ugly-hacks again. I moved to 14.04 LTS now, and since I found out Totem thumbnailer (or GStreamer more exactly?) started crashing for me at times, I moved to 'ffmpegthumbnailer' instead. It was also not too sensible to use the same filename all the time. Available at https://launchpad.net/~timo-jyrinki/+archive/shotwell Regarding dependencies and the feasibility of that one, it's not any better solution for Shotwell as such. It does not depend on many libraries, but it does on the libav which eg. in Ubuntu is in universe and I'm not sure if it's wanted to be promoted as GStreamer is there already. So again, I'm not proposing this as a solution, just a workaround. The remaining attachment could be marked obsolete too, but I don't have permissions to do so since it was imported from redmine.
Created attachment 293460 [details] [review] Use gdkpixbufsink in thumbnailer, from "git format-patch"
The above patch fixes shotwell-video-thumbnailer for me. It rips out a lot of the code, replacing it with the use of gdkpixbufsink.
Review of attachment 293460 [details] [review]: Very nice -- this has been a long time coming. Please commit and be sure to add yourself to the THANKS file.
I'm happy to see action here! I tried out the patch on an Ubuntu 14.04 LTS machine where I have my photo collection, but unfortunately I don't see a difference in my case. I reproduced the problem by using a USB drive with DCIM folder populated with the following AVCHD file from my Sony RX100 camera: http://kernelcat.com/shotwell/20141230125000.MTS Using this new patch [1] produced a gray thumbnail, while using my currently preferred solution of ffmpegthumbnailer [2] the video thumbnail is correct after importing. (Totem thumbnailer would work as well, Ubuntu just had a bug in April which prevented me from using it) I also tried a MP4 file importing which worked with and without new patches (ie the stock version). [1] https://launchpad.net/~timo-jyrinki/+archive/ubuntu/ppa/+sourcepub/4641871/+listing-archive-extra - shotwell-common + shotwell arch specific package [2] https://launchpad.net/~timo-jyrinki/+archive/ubuntu/shotwell/+sourcepub/4533252/+listing-archive-extra (the current patch version being http://kernelcat.com/shotwell/uglyhack_use_ffmpeg_thumbnailer.patch)
For easier comparison, here are the command lines: totem-video-thumbnailer -r 20141230125000.MTS totem.png /usr/lib/shotwell/shotwell-video-thumbnailer 20141230125000.MTS > shotwell.png ffmpegthumbnailer -s0 -i20141230125000.MTS -offmpeg.png Since Totem, Shotwell, and Shotwell via GdkPixbuf all use GStreamer, the only difference is that GStreamer is used in a different way leading to a seek that does not return a real full frame for some files but only a single interframe data (resulting in mostly gray).
(In reply to comment #8) > I'm happy to see action here! I tried out the patch on an Ubuntu 14.04 LTS > machine where I have my photo collection, but unfortunately I don't see a > difference in my case. I reproduced the problem by using a USB drive with DCIM > folder populated with the following AVCHD file from my Sony RX100 camera: > http://kernelcat.com/shotwell/20141230125000.MTS > > Using this new patch [1] produced a gray thumbnail, Really? This is what I get: http://cyberelk.net/tim/data/tmp/shotwell/20141230125000.png Maybe you're seeing a problem with that one because it's quite a short clip (approx 1s)? The seeking to find a thumbnail could be smarter, I'm sure. (In reply to comment #7) > Review of attachment 293460 [details] [review]: > > Very nice -- this has been a long time coming. Please commit and be sure to > add yourself to the THANKS file. Thanks. I don't seem to have commit access...
(In reply to comment #10) > Really? This is what I get: > http://cyberelk.net/tim/data/tmp/shotwell/20141230125000.png Great! And interesting, maybe it has something to do with older version of GStreamer (1.2.4). The patch itself was applied correctly. But, I can confirm with Ubuntu development version that with your patch I get correct thumbnails with Shotwell, even with the 1s clip. My findings just mean the fix can't be as easily backported to users of older libraries. > (In reply to comment #7) > > Review of attachment 293460 [details] [review] [details]: > > Very nice -- this has been a long time coming. Please commit and be sure to > > add yourself to the THANKS file. > Thanks. I don't seem to have commit access... I had, so since Jim gave +1 I submitted your patch at https://git.gnome.org/browse/shotwell/commit/?id=4d088d7bfd5dcf82d8f301667b3856764e7489ad Thanks a lot on behalf of what I believe are many users.
...and another commit for adding Tim to the THANKS file.
I'm closing this as fixed, as the purpose of this ticket was fulfilled by Tim's patch. To whit, this is not so much about making the Shotwell thumbnailer more compatible with video versions but to simplify it with newer GStreamer tools. If Tim's patch improves compatability and generates thumbnails for more video formats, that's gravy. If it doesn't, that's okay because the code is drastically simplified. Unfortunately, I went back and tested with some other recent videos and see that under newer versions of GStreamer this re-introduces bug #739396, which I'm re-opening. We do have outstanding tickets for improving thumbnails by searching for pixel variance (bug #716838), but this patch wasn't expected to solve that.
*** Bug 741493 has been marked as a duplicate of this bug. ***