GNOME Bugzilla – Bug 700610
Timeline clips filmstrip thumbnailer does too much background work and slows down / blocks the rest of the UI
Last modified: 2013-08-05 14:21:03 UTC
To reproduce in master: 1. Have "normal" computer hardware. Instead of a Core i7 with a solid-state drive, use a Core 2 Duo with a regular hard disk drive. However, the issue will still be noticeable even on the super-fast computer, just a little bit less so. We can't expect users to be running brand new computers bought last month, we should expect them to run machines that are five years old. 2. Have a project file with a bunch of clips in the timeline. Texture (from http://jeff.ecchi.ca/public/sample-pitivi-projects/ ) can be a good test case. 3. In ~/.cache/pitivi, rename or delete the "thumbs" folder. 4. Load the project. Result: - Project loading takes a LOT longer than before. It used to be nearly instantaneous before the implementation of the clutter timeline and new thumbnailer. - Extremely sluggish/non-responsive UI during (and after) project loading. You will feel it, guaranteed. On a Thinkpad X61, the system load goes to 14 when trying to load the “Texture” sample project. Yet, as far as we have been told, we are not blocking on I/O anymore (in theory), so maybe we are now starving the CPU by doing too much work at once and at the wrong time, preventing the UI from having enough ressources to be reactive. Would be nice to do profiling to be sure. The current thumbnailer implementation does two things: - It generates thumbnails for the visible portion of clips in the timeline, in higher priority - It does background processing to process the parts that are not currently visible Potential avenues to investigate or solve the problem: - Defer work to when nothing else is happening. Thumbnailing should never happen while we're loading a project, playing or rendering. Be able to pause and resume background thumbnailing. - Do less concurrent work. Throttle the background thumbnailing with a scheduler? Depending on the number of available CPU threads/cores. Maybe always try to leave one core free, especially if the pitivi window is not the focus (ie the user is doing something else)? Or, more simply, instead of processing all clips at the same time, perhaps limit to processing only one clip at a time for the background queue? - Previously, it committed (writes the clip's thumbnail cache sqlite file to disk) for every single thumbnail in the clip, which quickly destroyed performance by saturating with unnecessary I/O. The current implementation now commits only once the clip has been entirely processed, which is lighter in terms of I/O but also a bit overzealous, in the sense that it would be sufficient to commit using a timer (ex: every 30 seconds or when the clip processing completes, whichever comes first). Otherwise, if you try to thumbnail a clip that is two hours long, nothing will ever get written to disk until it has been thumbnailed entirely (which can take hours) Somewhat related (or perhaps to be obsoleted): https://bugzilla.gnome.org/show_bug.cgi?id=591427#c21
Created attachment 244710 [details] [review] An initial attempt by Daniel Thul This patch (not committed to master) made things a little bit better, but still noticeably laggy and we could still feel a slowdown.
Created attachment 244711 [details] [review] Another attempt by Daniel Thul Related to the first patch, Daniel then provided this other commit. However, this one makes things *much* worse.
I'm also wondering if it still makes sense to do background processing for non-visible portions at all - this means needing to add some weird throttling mechanisms to avoid overloading the machine, and I'm not sure we can actually guarantee that with the way gstreamer might (will?) use all available cores/threads to do video decoding anyway...
On my "thumbnailer-perf-hacks" branch on github: rebased/reworded Daniel's first commit, then added my own commit on top for the "autosave after a certain amount of time", which helps tremendously in avoiding wasted work. That doesn't solve the high CPU usage/concurrent processing issue, but at least mitigates part of the effects. Also interesting to note, KEY_UNIT seeks vs ACCURATE seeks makes no performance difference whatsoever with my camera's quicktime files (remains to be seen with less keyframe-intensive files) - whatsmore, thumbnailing doesn't work properly if using KEY_UNIT (you get empty spaces instead of the thumbnails on the clips). Another observation: there's a get_state in previewers.py, which is synchronous and thus probably a Bad Thing. According to Thibault, we should connect on the bus, and wait for the ASYNC_DONE message to be posted.
For the record, pushed the following two commits to master, which help in terms of "perceived" performance but don't solve the fundamental problem of doing too much work at once at the wrong time. Help wanted/needed for the rest of what I said above in the bug description. commit ddcee62ec0bbac753632e0a9888bd6ef8d0dffde Author: Jean-François Fortin Tam <nekohayo@gmail.com> Date: Mon May 27 22:36:32 2013 -0400 previewers: Periodically save clip thumbnails This improves perceived performance by saving "work in progress" thumbnailing. Otherwise, long clips might never have a saved thumbnail cache. commit e492dde5daccedbfeda9c3a1799f7d5b58676c8e Author: Daniel Thul <daniel.thul@gmail.com> Date: Mon May 27 22:30:30 2013 -0400 previewers: Avoid blocking the UI when generating clip thumbnails Put thumb generation in a low-priority idle GLib mainloop callback
I have implemented a class which will help us modulate the speed of the pipeline according to the current CPU usage here : https://github.com/MathieuDuponchelle/Pitivi/commit/68b30c4b1f599491a9cdc944d2ad17825d07ea8d
We have now committed a bunch of improvements (linearizing the processing as I suggested, in addition to modulating the processing speed depending on CPU usage) to help fix the remaining issues. It now works on an older laptop (Thinkpad X61) well-enough to not feel a terrible impact on playback and project loading, whether ~/.cache/pitivi exists or not. These are the core of the commits that implement this (there's a ton of lesser commits inbetween, so basically it's the stuff between commit ef71100478 and 26d4e73e). ----------- commit 26d4e73e8987c064524597f9eec5662c7cb4372c Author: Mathieu Duponchelle <mathieu.duponchelle@epitech.eu> Date: Tue Jul 30 18:25:33 2013 +0200 previewers: don't uselessly decode streams. commit c3e5ce6493c333f3b090aa3ef417e37054c75e08 Author: Thibault Saunier <thibault.saunier@collabora.com> Date: Sat Jul 27 07:25:39 2013 +0200 previewers: Make sure only 1 audio and 1 video pipeline run concurrently [...] (see commit message for lots of details) commit 3cdd2dbe2b087cf868fc6bcf1769941b8b609b55 Author: Thibault Saunier <thibault.saunier@collabora.com> Date: Sat Jul 27 01:15:16 2013 -0400 previewers: Only check messages from the level element to find infos about level We were parsing each and everyt message wish was useless commit b2ed619e5ea622a25e84e40ecbb1db511d3ec0ca Author: Jean-François Fortin Tam <nekohayo@gmail.com> Date: Tue Jul 2 00:53:57 2013 -0400 previewers: Default to 500 ms for the thumbnail interval This should make thumbnailing feel significantly faster on most computers commit 43255f6ed01ed11663a60cae7a467c3f78697640 Author: Jean-François Fortin Tam <nekohayo@gmail.com> Date: Tue Jul 2 00:39:05 2013 -0400 previewers: Delay initial thumbnail & waveform processing to when the UI is idle commit e3456b65f00e3d20bbb51efbfd0f29f389c4fcb6 Author: Mathieu Duponchelle <mathieu.duponchelle@epitech.eu> Date: Fri Jun 21 02:34:47 2013 +0200 Make the waveform pipelines go to READY when rate < 0.1. commit b04c2211119be42da6467c43c60f46bad8425776 Author: Mathieu Duponchelle <mathieu.duponchelle@epitech.eu> Date: Fri Jun 21 00:57:28 2013 +0200 Regulate the frequency at which thumbnails get created. + With respect to the CPU usage. commit 50546b07667617168ba6ee060e16a37f70fde156 Author: Mathieu Duponchelle <mathieu.duponchelle@epitech.eu> Date: Fri Jun 21 00:03:13 2013 +0200 previewers: add saving and loading for the waveforms. + They are pickled in ~/.cache/pitivi/waves commit ef71100478d8e1beb70ded00dee54e8ad83a8dbe Author: Mathieu Duponchelle <mathieu.duponchelle@epitech.eu> Date: Thu Jun 20 04:21:41 2013 +0200 previewers: implements a pipelineCPUAdapter class. + And use it in waveforms.