GNOME Bugzilla – Bug 510110
Support Amazon S3
Last modified: 2009-05-27 21:59:33 UTC
Support file sync using amazon S3. A very complete python S3 library is available http://code.google.com/p/boto/ (this bug was imported from trac)
See http://downloads.sourceforge.net/s4ync/s4ync-0.0.1.tar.gz
more example code http://code.google.com/p/boto/source/browse/trunk/s3put
Created attachment 114942 [details] AmazonS3Module.py Module for synchronizing with Amazon S3.
Created attachment 114943 [details] amazon.png Amazon logo from http://www.amazon.de/favicon.ico (contains three logos, extracted with "convert favicon.ico amazon.png").
Hi! Please find attached a module for synchronizing with Amazon S3. Unfortunately, it is not yet finished. 1. The configuration dialog needs to be improved. 2. The retrieval of a file from Amazon S3 seems to be buggy. I had a look at other data providers, and they retrieve a remote file by supplying their URL as an argument to File.File(). If I do that, I get the following error message: [datatypes.File ][WARNING] Cannot get info on non-existant file https://[removed].s3.amazonaws.com:443/bla?Signature=Lfl0I08123ACqiK04ByesqjTY%2Bc%3D&Expires=1216668548&AWSAccessKeyId=6CH73DHV5B2HB0I65393 (File.py:105) (see last comment in _get_data()). This is why I store the remote data in a local temporary file, which does not seem right. Maybe you can help me spot the mistake! Regards Christian
(In reply to comment #5) > Hi! > > Please find attached a module for synchronizing with Amazon S3. Unfortunately, > it is not yet finished. It looks good so far. Your coding style is excellent, and you seem to get all the main concepts in Conduit. > > 1. The configuration dialog needs to be improved. > 2. The retrieval of a file from Amazon S3 seems to be buggy. I had a look at > other data providers, and they retrieve a remote file by supplying their URL as > an argument to File.File(). If I do that, I get the following error message: > > [datatypes.File ][WARNING] Cannot get info on non-existant file > https://[removed].s3.amazonaws.com:443/bla?Signature=Lfl0I08123ACqiK04ByesqjTY%2Bc%3D&Expires=1216668548&AWSAccessKeyId=6CH73DHV5B2HB0I65393 > (File.py:105) > > (see last comment in _get_data()). This is why I store the remote data in a > local temporary file, which does not seem right. Maybe you can help me spot the > mistake! Have you considered ProxyFile? This allows you to supply all the information such as mtime and size, and it only uses the remote file when transferred. It should be more efficient, and it should prevent the errors you are seeing. I would also like to store a lot more information along with the file data. Like the mtime, file size, original location, relative path, etc. Good work! John > > Regards > Christian >
> (In reply to comment #5) > Have you considered ProxyFile? This allows you to supply all the information > such as mtime and size, and it only uses the remote file when transferred. It > should be more efficient, and it should prevent the errors you are seeing. Thanks for the input! I will have a look at it.
Can I be of any help with this?
Created attachment 115198 [details] proxyfiletest.py (In reply to comment #8) > Can I be of any help with this? Yes, your help would be greatly appreciated. Please find attached a sample script which has the following output on my computer: $ ./proxyfiletest.py Traceback (most recent call last):
+ Trace 203707
pf.transfer(expanduser('~/tmptestfile'))
data=cancel_function AccessDeniedError: Access denied 1: failure 2: success
The script first tries to get a file from Amazon S3 (filename 'a', filesize 0 bytes, link expires in about a week and works in Firefox) and fails. Then it tries to get another file from the web with exactly the same method, and succeeds. Maybe you can find out why it fails in the first case and why it works in the second case. Or maybe both links work on your computer and I just have the wrong libraries installed? Anyways, I would like to hear what you think about it.
Reported as bug #545000 to gnome-python. Let's see what they say...
Created attachment 116188 [details] AmazonS3Module.py Fixed some bugs, added glade configuration dialog. There are still two major problems: 1. Bug #545000: Has the possibility that it won't be fixed, as gnomevfs is deprecated. 2. http://code.google.com/p/boto/issues/detail?id=140
Created attachment 116189 [details] config.glade
I am working on a gio branch of conduit, and unfortunately the bug also exists there. Nevertheless I will commit this ASAP. See bug #547020
committed.