GNOME Bugzilla – Bug 634723
Hang when mass-removing/deleting clips from the timeline
Last modified: 2011-08-14 15:27:13 UTC
Created attachment 174365 [details] PITIVI_DEBUG=*:5 GST_DEBUG=2 In my "Boston summit" project, if I select half of the clips in the timeline and hit Delete (or use the toolbar button to remove the clips from the timeline), pitivi hangs. This doesn't happen if I delete clips one by one or in small amounts, only if I delete ~20 of them in one go.
Created attachment 174366 [details] output of "thread apply all bt" in gdb
Since this bug has apparently not attracted any notice, I can confirm it and add to it. For me the problem is MUCH more serious. I have a 1-hour video divided into about 20 clips that have been rearranged. On trying to delete several clips at a time, PiTiVi invariably hangs. Two adjacent clips is OK; three or more is problematic; five or more is an invitation to give up and start over. It also seems to help to wait a few seconds until the processor fan slows down to attempt the next deletion.
Created attachment 183225 [details] cProfile outputs Further observations: - it seems that the playhead seeks multiple times at the beginning of the timeline during the delete operation, you can see the previewer's image change repeatedly - it sometimes hangs, sometimes not (but takes a long time). When it doesn't hang, I was able to obtain a python profile output that I'll attach here. This sounds like a performance problem causing a race condition or something. I do wonder if solving the performance problem would fix everything, but at least it should alleviate part of the problem. -------- I've done some profiling on this, with multiple tries to ensure validity. The resulting graphs/profile outputs may have slightly different results, but generally the pattern is the same. What I can see is that when removeTrackObject is called: - The "remove" method of the gst.Bin objects is called about 900+ times (slightly strange given that I have less than 100 clips in the timeline). Anyway, this method seems very fast and only eats about 0.5% of the total time, so probably not a problem. - There is a really suspicious amount of calls to the "add" method of gst.Bin (1400+ times!) destroying performance, as we have seen in bug 591427 Not only that, but it does everything *twice*: - _updateDefaultSourcesUnchecked calls _getDefaultTRackObjectForStream 81 times which then calls makeBin which then calls "add" 180 times - _updateDefaultSourcesUnchecked calls "add" again at line 1033 Question: why does it even call "Add" in the first place, when we're doing a bunch of removes? From what I could understand of the code in track.py, it tries to add "default sources" for all the gaps in the timeline (note: my timeline doesn't even actually have any gaps between clips!)...
Created attachment 183226 [details] graphical profile of trial 1
Created attachment 183227 [details] graphical profile of trial 2
Created attachment 183228 [details] graphical profile of trial 3
Created attachment 183229 [details] graphical profile of simply loading the project For comparison purposes, this is the profile of only loading the project (not deleting any clips).
When it hangs on Ubuntu, the window is grayed out, and the system monitor shows python with a status "Sleeping", and using 0% CPU, so it's not even trying. Also, if I succeed in deleting the clips, then press Ctrl-z to undo the deletion, sometimes PiTiVi terminates. I don't know if that's a clue or a separate bug.
(In reply to comment #8) > Also, if I succeed in deleting the clips, then press Ctrl-z to undo the > deletion, sometimes PiTiVi terminates. I don't know if that's a clue or a > separate bug. That appears to be https://bugzilla.gnome.org/show_bug.cgi?id=636773# . The comment about sleeping still applies.
Created attachment 191602 [details] [review] Patch to fix mass deletion Thanks to Jean-Francois's profile, I was able to make a guess at what's causing the problem. The attached patch appears to resolve it correctly for me. I'm not sure if the patch will apply cleanly to mainline, since I developed it on my autoalign branch. The patch works by adding explicit methods for mass deletion, and delaying cleanup until after all the objects have been deleted. I must admit that, since I have not diagnosed the cause of the hang, I am not sure why this resolves it, and a race condition may still be lurking. However, I believe that this design is more sensible, and indeed the code seems to have been designed to enable this functionality, in which cleanup is delayed until after many objects have been deleted.
The above patch, rebased to master, is now available from https://github.com/bemasc/pitivi/commits/massdelete
As thiblahute noted, this patch solves the deletion case, but the bug is still triggered if you Undo the deletion, presumably because adding large numbers of objects separately runs into the same race condition as deleting large numbers of objects. There are several potential strategies to solve this ... one of which would be to resolve the underlying race condition! Alternatively, we could delay a Track's cleanup (Track._update*) operations until the end of an Undo block, which might also make sense for the sake of efficiency.
This problem has been fixed in the development version. The fix will be available in the next major software release. Thanks a lot for your help. Relevant commits: 5842b and b2ce2
To be clear; the applied patches fix the bug during deletion, but the same problem is still triggered if you Undo the operation, so I am not sure this bug is really "FIXED".