After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 700610 - Timeline clips filmstrip thumbnailer does too much background work and slows down / blocks the rest of the UI
Timeline clips filmstrip thumbnailer does too much background work and slows ...
Status: RESOLVED FIXED
Product: pitivi
Classification: Other
Component: Timeline
Git
Other Linux
: Normal major
: 0.91
Assigned To: Pitivi maintainers
Pitivi maintainers
Depends on:
Blocks:
 
 
Reported: 2013-05-18 23:27 UTC by Jean-François Fortin Tam
Modified: 2013-08-05 14:21 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
An initial attempt by Daniel Thul (1.30 KB, patch)
2013-05-19 13:29 UTC, Jean-François Fortin Tam
needs-work Details | Review
Another attempt by Daniel Thul (1.99 KB, patch)
2013-05-19 13:30 UTC, Jean-François Fortin Tam
rejected Details | Review

Description Jean-François Fortin Tam 2013-05-18 23:27:50 UTC
To reproduce in master:
1. Have "normal" computer hardware. Instead of a Core i7 with a solid-state drive, use a Core 2 Duo with a regular hard disk drive. However, the issue will still be noticeable even on the super-fast computer, just a little bit less so. We can't expect users to be running brand new computers bought last month, we should expect them to run machines that are five years old.

2. Have a project file with a bunch of clips in the timeline. Texture (from http://jeff.ecchi.ca/public/sample-pitivi-projects/ ) can be a good test case.

3. In ~/.cache/pitivi, rename or delete the "thumbs" folder.

4. Load the project.



Result:
- Project loading takes a LOT longer than before. It used to be nearly instantaneous before the implementation of the clutter timeline and new thumbnailer.
- Extremely sluggish/non-responsive UI during (and after) project loading. You will feel it, guaranteed.

On a Thinkpad X61, the system load goes to 14 when trying to load the “Texture” sample project.


Yet, as far as we have been told, we are not blocking on I/O anymore (in theory), so maybe we are now starving the CPU by doing too much work at once and at the wrong time, preventing the UI from having enough ressources to be reactive. Would be nice to do profiling to be sure.

The current thumbnailer implementation does two things:
- It generates thumbnails for the visible portion of clips in the timeline, in higher priority
- It does background processing to process the parts that are not currently visible



Potential avenues to investigate or solve the problem:

- Defer work to when nothing else is happening. Thumbnailing should never happen while we're loading a project, playing or rendering. Be able to pause and resume background thumbnailing.

- Do less concurrent work. Throttle the background thumbnailing with a scheduler? Depending on the number of available CPU threads/cores. Maybe always try to leave one core free, especially if the pitivi window is not the focus (ie the user is doing something else)? Or, more simply, instead of processing all clips at the same time, perhaps limit to processing only one clip at a time for the background queue?

- Previously, it committed (writes the clip's thumbnail cache sqlite file to disk) for every single thumbnail in the clip, which quickly destroyed performance by saturating with unnecessary I/O. The current implementation now commits only once the clip has been entirely processed, which is lighter in terms of I/O but also a bit overzealous, in the sense that it would be sufficient to commit using a timer (ex: every 30 seconds or when the clip processing completes, whichever comes first). Otherwise, if you try to thumbnail a clip that is two hours long, nothing will ever get written to disk until it has been thumbnailed entirely (which can take hours)


Somewhat related (or perhaps to be obsoleted): https://bugzilla.gnome.org/show_bug.cgi?id=591427#c21
Comment 1 Jean-François Fortin Tam 2013-05-19 13:29:42 UTC
Created attachment 244710 [details] [review]
An initial attempt by Daniel Thul

This patch (not committed to master) made things a little bit better, but still noticeably laggy and we could still feel a slowdown.
Comment 2 Jean-François Fortin Tam 2013-05-19 13:30:53 UTC
Created attachment 244711 [details] [review]
Another attempt by Daniel Thul

Related to the first patch, Daniel then provided this other commit. However, this one makes things *much* worse.
Comment 3 Jean-François Fortin Tam 2013-05-26 23:25:38 UTC
I'm also wondering if it still makes sense to do background processing for non-visible portions at all - this means needing to add some weird throttling mechanisms to avoid overloading the machine, and I'm not sure we can actually guarantee that with the way gstreamer might (will?) use all available cores/threads to do video decoding anyway...
Comment 4 Jean-François Fortin Tam 2013-05-28 03:49:24 UTC
On my "thumbnailer-perf-hacks" branch on github: rebased/reworded Daniel's first commit, then added my own commit on top for the "autosave after a certain amount of time", which helps tremendously in avoiding wasted work. That doesn't solve the high CPU usage/concurrent processing issue, but at least mitigates part of the effects.

Also interesting to note, KEY_UNIT seeks vs ACCURATE seeks makes no performance difference whatsoever with my camera's quicktime files (remains to be seen with less keyframe-intensive files) - whatsmore, thumbnailing doesn't work properly if using KEY_UNIT (you get empty spaces instead of the thumbnails on the clips).

Another observation: there's a get_state in previewers.py, which is synchronous and thus probably a Bad Thing. According to Thibault, we should connect on the bus, and wait for the ASYNC_DONE message to be posted.
Comment 5 Jean-François Fortin Tam 2013-06-08 18:28:00 UTC
For the record, pushed the following two commits to master, which help in terms of "perceived" performance but don't solve the fundamental problem of doing too much work at once at the wrong time. Help wanted/needed for the rest of what I said above in the bug description.


commit ddcee62ec0bbac753632e0a9888bd6ef8d0dffde
Author: Jean-François Fortin Tam <nekohayo@gmail.com>
Date:   Mon May 27 22:36:32 2013 -0400

 previewers: Periodically save clip thumbnails
    
 This improves perceived performance by saving "work in progress" thumbnailing.
 Otherwise, long clips might never have a saved thumbnail cache.


commit e492dde5daccedbfeda9c3a1799f7d5b58676c8e
Author: Daniel Thul <daniel.thul@gmail.com>
Date:   Mon May 27 22:30:30 2013 -0400

    previewers: Avoid blocking the UI when generating clip thumbnails
    
    Put thumb generation in a low-priority idle GLib mainloop callback
Comment 6 Mathieu Duponchelle 2013-06-20 02:26:14 UTC
I have implemented a class which will help us modulate the speed of the pipeline according to the current CPU usage here :

https://github.com/MathieuDuponchelle/Pitivi/commit/68b30c4b1f599491a9cdc944d2ad17825d07ea8d
Comment 7 Jean-François Fortin Tam 2013-08-05 14:21:03 UTC
We have now committed a bunch of improvements (linearizing the processing as I suggested, in addition to modulating the processing speed depending on CPU usage) to help fix the remaining issues. It now works on an older laptop (Thinkpad X61) well-enough to not feel a terrible impact on playback and project loading, whether ~/.cache/pitivi exists or not.

These are the core of the commits that implement this (there's a ton of lesser commits inbetween, so basically it's the stuff between commit ef71100478 and 26d4e73e).


-----------
commit 26d4e73e8987c064524597f9eec5662c7cb4372c
Author: Mathieu Duponchelle <mathieu.duponchelle@epitech.eu>
Date:   Tue Jul 30 18:25:33 2013 +0200

    previewers: don't uselessly decode streams.


commit c3e5ce6493c333f3b090aa3ef417e37054c75e08
Author: Thibault Saunier <thibault.saunier@collabora.com>
Date:   Sat Jul 27 07:25:39 2013 +0200

    previewers: Make sure only 1 audio and 1 video pipeline run concurrently

    [...] (see commit message for lots of details)


commit 3cdd2dbe2b087cf868fc6bcf1769941b8b609b55
Author: Thibault Saunier <thibault.saunier@collabora.com>
Date:   Sat Jul 27 01:15:16 2013 -0400

    previewers: Only check messages from the level element to find infos about level
    
    We were parsing each and everyt message wish was useless


commit b2ed619e5ea622a25e84e40ecbb1db511d3ec0ca
Author: Jean-François Fortin Tam <nekohayo@gmail.com>
Date:   Tue Jul 2 00:53:57 2013 -0400

    previewers: Default to 500 ms for the thumbnail interval
    
    This should make thumbnailing feel significantly faster on most computers


commit 43255f6ed01ed11663a60cae7a467c3f78697640
Author: Jean-François Fortin Tam <nekohayo@gmail.com>
Date:   Tue Jul 2 00:39:05 2013 -0400

    previewers: Delay initial thumbnail & waveform processing to when the UI is idle


commit e3456b65f00e3d20bbb51efbfd0f29f389c4fcb6
Author: Mathieu Duponchelle <mathieu.duponchelle@epitech.eu>
Date:   Fri Jun 21 02:34:47 2013 +0200

    Make the waveform pipelines go to READY when rate < 0.1.

commit b04c2211119be42da6467c43c60f46bad8425776
Author: Mathieu Duponchelle <mathieu.duponchelle@epitech.eu>
Date:   Fri Jun 21 00:57:28 2013 +0200

    Regulate the frequency at which thumbnails get created.
    
        + With respect to the CPU usage.

commit 50546b07667617168ba6ee060e16a37f70fde156
Author: Mathieu Duponchelle <mathieu.duponchelle@epitech.eu>
Date:   Fri Jun 21 00:03:13 2013 +0200

    previewers: add saving and loading for the waveforms.
    
        + They are pickled in ~/.cache/pitivi/waves


commit ef71100478d8e1beb70ded00dee54e8ad83a8dbe
Author: Mathieu Duponchelle <mathieu.duponchelle@epitech.eu>
Date:   Thu Jun 20 04:21:41 2013 +0200

    previewers: implements a pipelineCPUAdapter class.
    
        + And use it in waveforms.