After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 668028 - Ability to export a project and associated media files as a tarball
Ability to export a project and associated media files as a tarball
Status: RESOLVED FIXED
Product: pitivi
Classification: Other
Component: General
Git
Other Linux
: Normal enhancement
: 0.91
Assigned To: Paul Lange
Pitivi maintainers
Depends on:
Blocks:
 
 
Reported: 2012-01-16 15:51 UTC by Jean-François Fortin Tam
Modified: 2012-04-02 19:48 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
first patch (5.46 KB, patch)
2012-03-28 06:13 UTC, Paul Lange
none Details | Review
second round (5.46 KB, patch)
2012-03-30 05:16 UTC, Paul Lange
none Details | Review
correct version 2 (5.46 KB, patch)
2012-03-30 05:19 UTC, Paul Lange
none Details | Review
improved export (7.06 KB, patch)
2012-03-31 18:40 UTC, Paul Lange
none Details | Review

Description Jean-François Fortin Tam 2012-01-16 15:51:37 UTC
I want a "Project > Export as archive" menu item that does the following:
- check for available hard drive space
- show a filechooser dialog requesting where to save the tarball
- create a .tar archive
- for each source in the media library: add it to the tarball (try preserving paths/preventing filename collisions?). During this process, show a progressbar.
- add the xptv to the tarball

Not only is this useful for backups or for sharing projects with collaborators, but this is really needed for testing/debugging.

Bonus points if your implementation offers (optionally) to use clip proxies (see bug 609136) to make the file size smaller (for easier upload).
Comment 1 bens 2012-01-16 16:39:39 UTC
It might be worth checking if gio/gvfs/libarchive would allow such a file to be processed without having to maintain any archive handling code inside pitivi.

Gvfs definitely did have (read-only) transparent archive reading code at one point, but I don't know what the status is now.
Comment 2 Paul Lange 2012-03-28 06:13:21 UTC
Created attachment 210758 [details] [review]
first patch

I created a patch for this problem which applies on the ges branch. Right now it doesn't include a progressbar to show progress but that should be doable if you agree with the basic ideas.

One problem I see is where to activate the "export" action. Right now I activate it always but it should only be activated if there's a project to export.
Comment 3 Paul Lange 2012-03-30 05:16:27 UTC
Created attachment 210918 [details] [review]
second round

Second iteration, addresses the following things mentioned in IRC:
- default archive name based on projects name
- change default ending of the project file to xptv
- renamed action to "Export as Archive"
- correctly activate Export action
- save project file in root directory of tar-file
Comment 4 Paul Lange 2012-03-30 05:19:27 UTC
Created attachment 210919 [details] [review]
correct version 2
Comment 5 Jean-François Fortin Tam 2012-03-31 03:15:03 UTC
There's this line:

>            self.actiongroup.get_action("ExportProject").set_sensitive(True)


...which is in the wrong place. It is inside an if condition that is only triggered if the project you're loading had missing clips in it. Putting that line outside the if clause works.

After that, I did some quick smoke-testing. It works! (except the project.name thing for the .xptv, but that might not be your patch being wrong, probably the ges pitivi formatter metadata).

Actually the menu item string should have been "Export as Archive..." instead of "Export as Archive", my mistake! The "..." are meant to indicate that further user interaction is expected (ex: specifying where to export).

You should probably check tarfile.open for IOError exceptions. Just in case someone tries to save in / or a read-only folder :)


Some thoughts/questions:
For media files, it recreates the whole directory structure from "/". For example it creates /home/jeff/foo/bar/baz.ogg inside the tarball. How do you feel about it? I'm wondering if we could do some magic where you compare all the paths of the project media and remove the common part of it, then put the result in a "media" subfolder of the tarball (ex: if I have a file in foo and a file in bar, create only "foo" in the tarball, not /home/jeff/foo), but I'm not sure how complex and robust that'd be.

I'm also wondering if the root of the tarball should be contained in a folder to prevent potential conflicts on the user's filesystem, but maybe I'm overthinking it.

All in all: awesome.

Other than that, do you have outstanding things to be done with your patch? For the final iteration, you should make your patch with git format-patch (you can use gitg, which makes this easy).
Comment 6 Paul Lange 2012-03-31 18:40:01 UTC
Created attachment 211038 [details] [review]
improved export

Thanks for the review!

Reworked quite a bit to add some stuff:
 - changed action label
 - corrected activation of action
 - it now creates a top-level dir in the tar-file
 - check for exceptions in the tar-file creation code
 - if all sources are located in the users home dir, don't save it to the tar-file

The projects name is used if available, but since GES doesn't save it right now it usually doesn't show up when testing.
Comment 7 Jean-François Fortin Tam 2012-04-02 16:03:11 UTC
Your reworked version brings up this question: have you seen the methods for handling uris and paths etc. in the utils module?

I remember fighting a lot with uri encoding recently to make it robust/standard, so take a look if there might be an existing solution instead of manually concatenating file:// and such. If so, it might be more robust.
Comment 8 Jean-François Fortin Tam 2012-04-02 19:19:03 UTC
For the record: the latest iteration of the patch works and seems OK to me. If anyone has further comments, please say so before I suddenly decide to merge this ;)
Comment 9 Thibault Saunier 2012-04-02 19:34:05 UTC
Ok, seems almost ready, you should just check the dependency (tarfile) and make it optionnal in your patch
Comment 10 Jean-François Fortin Tam 2012-04-02 19:48:11 UTC
tarfile is a standard python module, so it's fine on that end. Will open a new feature request on GES to have that functionality there instead of in pitivi. In the meantime, pushed:


commit be6cb03958672cf2e4bd742d052ba004b0a99999
Author: Paul Lange <palango@gmx.de>
Date:   Tue Mar 27 12:27:00 2012 -0600

    Add basic export functionality
    
    This commit adds a new action to export a project to a tar-file.
    This tar-file includes the project file and all source files. So
    far no compression is used but could be added easily.