GNOME Bugzilla – Bug 609377
waveforms morph/change/are regenerated when seeking
Last modified: 2013-09-19 21:27:57 UTC
I'll attach a screencast, this is really strange. When the playhead moves, sometimes the waveforms under it are transformed. This tells me two things: - there is something flawed in the way they are calculated, since they are calculated differently the second time - there might be an ugly iceberg underneath this. Why is it "showing" this symptom? Doesn't that mean that it somehow tries to recalculate waveforms *all the time* when the playhead moves? If so, isn't that a performance problem? By the way, this also happens when rendering (since the playhead moves by itself at that time). It may be more apparent with certain waveforms/clips than others.
Screencast: http://jeff.ecchi.ca/public/pitivi-609377.ogv This also raises doubt in the user's mind: the user can't "trust" waveforms' if they change all the time: "which version is the correct one?"... so tagging with keyword "usability".
I have some idea what's going on, and it's not seeking per se. You will also get this bug if you damage different parts of the preview image (select, unselect, modify alpha property, etc). The reason it's connected with seeking is that the play-head moves over the clip and thus triggers an update of the clip. What's going on is that the previewer sometimes uses a different thumbnail image for the same region fo the clip. I think this has do with the way timestamps are calculated. Basically we index every thumbnail tile by its timestamp, and we get the time-stamp based on the visual position of the clip that needs to be re-drawn. The drawing algorithm is particularly sensitive to the timestamp used (because it works by taking min/max values over a group of samples), so plus or minus a few samples can have a large impact on the shape of the waveform. The calculation involves floating-point arithmetic, including divisions and modular arithmetic. There might be a bug in the code, or perhaps something more subtle like a rounding error. This this leads to two distinct timestamps for essentially the same image. I have tried to make the code robust by quantizing all timestamps to the nearest frame, but I have not been able to entirely eliminate this issue.
I had a closer look at this and it seems that the first part always gets updated correctly when the playhead passes it (I *guess* that's the first thumbnail). All the following thumbnail updates seem to be in the right order, but shifted leftwards by the length of one thumbnail. Assuming that the part at the beginning, which gets updated correctly, is thumbnail 1, it would look like this: thumbnail | 1 | 2 | 3 | 4 | ... --------------------------------------------- updated with thumbnail | 1 | 1 | 2 | 3 | ...
Created attachment 155689 [details] screenshots of bug 609377
This error is caused by the lacking precision of the timestamps calculated by ui.zoominterface.Zoomable.pixelToNs() A real example: =============== cls.zoomratio = 5.16177175: twidth, the width of the audio thumbnails is calculated in ui.previewer.RandomAccessAudioPreviewer() via ui.zoominterface.Zoomable.nsToPixel(cls, 30000000000) twidth = 154 pixel (integer) Now ui.previewer.RandomAccessPreviewer.render_cairo wants to render a part between bounds.x1: 154.0 , bounds.x2: 162.0 (an area in the second thumbnail) and correctly finds that the first pixel of the thumbnail is at i = 154. Until here everthing is OK But then it calculates the timestamp j with j = Zoomable.pixelToNs(i - sof) # sof was 0 in this example j = 29834717120 j would be 30000000000 (start of the second thumbnail) if we had infinite precision. Notice that 29834717120 is a timestamp inside the first thumbnail! Now the method RandomAccessAudioPreviewer._segment_for_time(self, time): # for audio files, we need to know the duration the segment spans return time - (time % self.tdur), self.tdur returns '(0, 30000000000)' instead of '(30000000000, 30000000000)' (which is what we would need) and the first thumbnail is used to render the area of the second thumbnail. =============== If you use the middle of the thumbnail instead of the left edge for calculating timestamps the effect can be reduced, but it still occurs when the clips extend a certain length.
The last sentence wasn't correct. I meant to say that tweaking j = Zoomable.pixelToNs(i - sof) to something like j = Zoomable.pixelToNs(i - sof) + self.tdur/2 helps a litte, but it doesn't eliminate the problem as a whole, it just makes it occurs later in the track ...
Thanks for that analysis, that will help out a lot.
Created attachment 155922 [details] [review] Not a patch, just for the record Just a quick hack, to demonstrate how restraining the values of zommratio to 0.1, 0.2, .... , 0.9, 1, 2, 3 ... makes pixel -> time calculations more solid (and zooming less smooth ;)).
Created attachment 156008 [details] [review] Idea of a solution Idea: Don't just use cls.zoomratio for calculating time positions in thumbs from pixels, but enable RandomAccessPreviewer to calculate pixel => time based on the *actual* pixel/second ratio of its thumbnails.
Created attachment 156018 [details] [review] Undo using timestamps of the middle of the thumbs OK, using timestamps of the middle of the thumbnails was a bit of overdoing, as they are used for generating the video thumbs, take that part back ...
Review of attachment 156018 [details] [review]: No good! Waveforms and audio out of sync now.
I reported this as https://bugs.launchpad.net/ubuntu/+source/pitivi/+bug/578530 before discovering this Pitivi bugzilla database. Note that the launchpad bug also reports that the waveform and audio are about 15s out of sync at the end of a 60 minute audio track.
The reason for this bug is that ptivi has an unreliable way to convert time to pixels and the other way round, see https://bugzilla.gnome.org/show_bug.cgi?id=613064.
I just hit this bug as well, trying to sync a MP4's internal AAC audio, with an externally captured WAV (for better quality). The waveform for the .WAV file did not seem to morph...
*** Bug 694642 has been marked as a duplicate of this bug. ***
This problem has been fixed in the development version. The fix will be available in the upcoming alpha release.