After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 591427 - Thumbnails slow down the insertion of sources/clips to the timeline
Thumbnails slow down the insertion of sources/clips to the timeline
Status: RESOLVED FIXED
Product: pitivi
Classification: Other
Component: General
Git
Other Linux
: Normal major
: 0.91
Assigned To: René Stadler
Pitivi maintainers
: 610554 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2009-08-11 09:41 UTC by Edward Hervey
Modified: 2013-08-05 14:07 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Full breakdown of newprojectloadedCb (4.49 KB, text/plain)
2009-09-06 09:24 UTC, Edward Hervey
  Details
cProfile output of pitivi loading a project (542.61 KB, application/octet-stream)
2010-11-23 20:46 UTC, Jean-François Fortin Tam
  Details
graph of the cProfile output (382.96 KB, image/png)
2010-11-23 20:47 UTC, Jean-François Fortin Tam
  Details
factories: First round at making linking faster (5.32 KB, patch)
2010-11-24 17:06 UTC, Edward Hervey
none Details | Review
pitivi: More linking speedups (3.70 KB, patch)
2010-11-24 18:08 UTC, Edward Hervey
none Details | Review
cProfile output of pitivi loading a project, with patches 1 and 2 (548.40 KB, application/octet-stream)
2010-11-24 21:14 UTC, Jean-François Fortin Tam
  Details
graph of the cProfile output, with patches 1 and 2 (467.49 KB, image/png)
2010-11-24 21:15 UTC, Jean-François Fortin Tam
  Details

Description Edward Hervey 2009-08-11 09:41:05 UTC
Import some photos in the source list (say 30 or so photos)
Add them to the timeline (by selecting them all (CTRL+A) and inserting them (INSERT)) => takes a loooong time whereas it shouldn't.
Comment 1 Edward Hervey 2009-08-11 09:52:04 UTC
Had a quick look, it's not the gst_bin_add(composition, sources) which is taking that time (it only takes a few milliseconds per added source).
Comment 2 Edward Hervey 2009-09-06 09:23:51 UTC
It's the Preview (thumbnails/waveforms) which is the culprit.

This is the result of running the hotshot python profiler on MainWindow::_newProjectLoadedCb with a project containing 56 photos

Total Execution time : 6.405s

TrackObject.__init__() : 5.927s  (94% of total)
 Preview.__init__() : 5.512s (86% of total)
  RandomAccessPreviewer.__init__() : 5.462s (85% of total)
   RandomAccessPreviewer._pipelineInit() : 5.206s (81% of total)

And we can break down _pipelineInit() into two parts:
pitivi.utils.pipeline : 3.138s (60% of _pipelineInit)
SingleDecodeBin.do_change_state : 2.013s (39% of _pipelineInit)

Oh, Did I mention this was with thumbnailing/waveforms disabled ???
Comment 3 Edward Hervey 2009-09-06 09:24:58 UTC
Created attachment 142584 [details]
Full breakdown of newprojectloadedCb

This is the full breakdown of cpu usage sorted by "cumulative time usage" (which gives you a linear breakdown of cpu usage).
Comment 4 Edward Hervey 2009-09-06 09:27:03 UTC
And for those wondering, This is the link to the docs of the hotshot profiler:
http://docs.python.org/library/hotshot.html#module-hotshot
Comment 5 Edward Hervey 2009-09-06 09:36:16 UTC
Forgot to put my remarks on how to improve the situation :)

The problem is that the pipeline of the previewer is created AND set to PAUSED in the initialization. There's seriously no use for that.

* It should only be created *IF* needed (I had thumbnailing/waveform disabled),
* It should only be created *WHEN* needed (i.e. not when creating the previewer, but when the first thumbnail/waveform is requested)
* It could maybe be set to PAUSED asynchronously (in a separate thread).
* Don't activate all previewer pipelines at once.
Comment 6 Stephen Griffiths 2010-02-16 15:45:24 UTC
I beleive this was fixed in git with the new release of gstreamer. marking RESOLVED, FIXED.
Comment 7 Jean-François Fortin Tam 2010-02-17 22:50:24 UTC
Nope, don't confuse "importing into the source list" and "inserting into the timeline" (which is what this bug is about, AFAICT).
Comment 8 Edward Hervey 2010-03-10 18:42:12 UTC
Bumping to next milestone :(
Comment 9 Jean-François Fortin Tam 2010-05-25 19:59:56 UTC
*** Bug 610554 has been marked as a duplicate of this bug. ***
Comment 10 Jean-François Fortin Tam 2010-09-09 19:47:32 UTC
Bumping to next milestone ;(
Comment 11 Jean-François Fortin Tam 2010-11-23 20:46:00 UTC
Created attachment 175129 [details]
cProfile output of pitivi loading a project

Here's the output of 
python -m cProfile -o foo bin/pitivi


Then, you can get some nice stats:


jeff@kusanagi:~$ python
>>> import pstats
>>> p = pstats.Stats("profile - load project Texture")

# Then run the following two commands to get some nice stats:
>>> p.strip_dirs().sort_stats("cumulative").print_stats(20)
>>> p.strip_dirs().sort_stats("calls").print_stats(20)

There are methods that are called over 80 000 times in there...
Comment 12 Jean-François Fortin Tam 2010-11-23 20:47:57 UTC
Created attachment 175130 [details]
graph of the cProfile output

And here is a nice graph generated from the previous output. It was done using http://gprof2dot.jrfonseca.googlecode.com/hg/gprof2dot.py

By running

python gprof2dot.py -f pstats "foo"  | dot -Tpng -o output.png
Comment 13 Jean-François Fortin Tam 2010-11-24 04:33:30 UTC
- GST's "link" method is used in factories/test.py's _makeStreamBin and _makeStreamBinReal methods

- GST's "element_link_many" method is used in factories/base.py's _addCommonVideoElements method (perhaps in _addCommonAudioElements too, but my test case probably didn't have enough audio clips so it didn't show up).



By looking at the graph and from a theoretical point of view with my limited understanding, I'm not entirely surprised to hypothesize that linking is the problem here (not drawing on the timeline canvas nor thumbnails/waveforms). But I may be completely wrong..?
Comment 14 Edward Hervey 2010-11-24 10:36:05 UTC
All of those are caused by heavy caps negotiation whenever doing a link.

We should switch to using gst.Pad.link_full or gst.Element.link_pads_full with the gst.PAD_LINK_CHECK_NOTHING (or PAD_LINK_CHECK_TEMPLATE_CAPS argument to avoid unneeded caps negotiation at link time.

WARNING : you need to be *sure* those elements are compatible if you use the LINK_CHECK_NOTHING argument, else the link will be accepted, but it will fail later on during the actual negotiation (when the first buffer comes through).

I thought we had fixed it for the real bottlenecks.... apparently not :(
Comment 15 Edward Hervey 2010-11-24 17:06:42 UTC
Created attachment 175187 [details] [review]
factories: First round at making linking faster
Comment 16 Jean-François Fortin Tam 2010-11-24 17:30:25 UTC
Initial warm-cache tests with patch #175187 :

boston summit project (patched): 2.5 seconds
boston summit project (unpatched): 7.9 seconds

texture project (unpatched): 1 min 32.4 seconds
texture project (patched): 31.4 seconds

(Insertion times exclude the time to import clips to the source list)
Comment 17 Edward Hervey 2010-11-24 18:08:29 UTC
Created attachment 175193 [details] [review]
pitivi: More linking speedups
Comment 18 Jean-François Fortin Tam 2010-11-24 21:14:50 UTC
Created attachment 175201 [details]
cProfile output of pitivi loading a project, with patches 1 and 2

Patch #175193 does not significantly change performance for me after the initial patch. Here is an updated cProfile output with the two patches applied.
Comment 19 Jean-François Fortin Tam 2010-11-24 21:15:49 UTC
Created attachment 175202 [details]
graph of the cProfile output, with patches 1 and 2
Comment 20 Edward Hervey 2010-11-25 09:40:18 UTC
Committed the two patches but keeping the bug open so we can profile more what's going on.

commit e276e4656eb3dae4d84cf4775217e9d9a48e0ea1
Author: Edward Hervey <bilboed@bilboed.com>
Date:   Wed Nov 24 19:08:09 2010 +0100

    pitivi: More linking speedups
    
    https://bugzilla.gnome.org/show_bug.cgi?id=591427

commit 13d285d781849b43ed3d802d4b2aa8155a6fb60a
Author: Edward Hervey <bilboed@bilboed.com>
Date:   Wed Nov 24 18:04:31 2010 +0100

    factories: First round at making linking faster
    
    https://bugzilla.gnome.org/show_bug.cgi?id=591427
Comment 21 Jean-François Fortin Tam 2012-07-10 20:43:33 UTC
Actually, from comparing with/without thumbnails.py, the only part slowing things down nowadays (in the GES version) is the thumbnailing code, which was absent in d228a0c but re-added right before the merge of the ges branch (e04a3d).

It seems some part of thumbnail processing still happens on startup even if the user has disabled thumbnailing in the preferences. Two things tell me this:

1) loading/populating a timeline with multiple clips on multiple layers is much slower than in d228a0c... the slowness is the same as what we were seeing in pitivi 0.15

2) when running with gst 0.10.35, there was a warning hinting at the fact that the thumbnaile’s pipeline init (and surely many other methods) is called on startup (this warning does not occur with gst 0.10.36+, but at least it was a good indicator):

Warning: value "((GstVideoScaleMethod) 3)" of type `GstVideoScaleMethod' is invalid or out of range for property `method' of type `GstVideoScaleMethod'


The thumbnail is not fully async: while the timeline is being created, the UI is unresponsive (as stated above), and similarly so when zooming into multiple clips on different layers.
Comment 22 Jean-François Fortin Tam 2013-08-05 14:07:54 UTC
Pretty much solved nowadays. See also bug #700610.