After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 546591 - File.copy progress_callback does not work
File.copy progress_callback does not work
Status: RESOLVED FIXED
Product: pygobject
Classification: Bindings
Component: gio
Git master
Other Linux
: High critical
: ---
Assigned To: Nobody's working on this now (help wanted and appreciated)
Python bindings maintainers
: 546601 (view as bug list)
Depends on: 547021
Blocks: 547022 569774
 
 
Reported: 2008-08-06 13:38 UTC by John Stowers
Modified: 2009-02-15 22:24 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
fix? (1.87 KB, patch)
2008-08-09 13:47 UTC, Paul Pogonyshev
committed Details | Review

Description John Stowers 2008-08-06 13:38:12 UTC
Using SVN Head.

import gio

s = gio.File("a.txt")
d = gio.File("b.txt")

canc = gio.Cancellable()

def progress(current, total, cancellable):
    print "%2.1f%%" % current/total

ok = s.copy(
        destination=d,
        flags=gio.FILE_COPY_NONE,
        cancellable=canc,
        progress_callback=progress,
        user_data=canc
        )

print "OK? %s" % ok

Gives:

TypeError: callback argument not callable
Comment 1 Johan (not receiving bugmail) Dahlin 2008-08-06 14:01:03 UTC
Tricky to fix. As a workaround, don't pass in cancellable as user_data.
Comment 2 John Stowers 2008-08-06 14:20:43 UTC
Gives the same result

ok = s.copy(
        destination=d,
        flags=gio.FILE_COPY_NONE,
        cancellable=canc,
        progress_callback=progress
        )

Traceback (most recent call last):
  • File "progress-cb.py", line 15 in <module>
    progress_callback=progress
TypeError: callback argument not callable

Comment 3 Johan (not receiving bugmail) Dahlin 2008-08-06 14:27:18 UTC
Oh, I fixed that part locally already. So if you update to svn it'll work.
It worked before that, but only with positional arguments, not keyword arguments.
Comment 4 Johan (not receiving bugmail) Dahlin 2008-08-06 14:59:21 UTC
*** Bug 546601 has been marked as a duplicate of this bug. ***
Comment 5 Johan (not receiving bugmail) Dahlin 2008-08-06 15:01:07 UTC
bug 546601 is the same bug and I know why it happens:
The notify (which contains the callback and the user data) is freed in the end of the progress callback, the second time it's called it accessed invalid memory.

To solve this we need a GDestroyNotify, perhaps gio needs to be extended for that to happen.
Comment 6 John Stowers 2008-08-06 21:35:36 UTC
Can you think of a way to work around this? 

This functionality is necessary for Conduit to move to GIO.
Comment 7 Johan (not receiving bugmail) Dahlin 2008-08-06 22:05:01 UTC
Dunno.
Maybe we need to get some patches into GIO.
Comment 8 Matthias Clasen 2008-08-09 03:20:27 UTC
> The notify (which contains the callback and the user data) is freed in the end
> of the progress callback, the second time it's called it accessed invalid
> memory.

Whats the second time here ? 

Seems to me that refcounting should be sufficient to solve this. Just ref the cancellable once for each callback it is passed to, and unref it when the callback is done...
Comment 9 Paul Pogonyshev 2008-08-09 13:14:55 UTC
> The notify (which contains the callback and the user data) is freed in the end
> of the progress callback, the second time it's called it accessed invalid
> memory.

Documentation says:
> It is guaranteed that this callback will be called after all data has been
> transferred with the total number of bytes copied during the operation.

Shouldn't we then free 'notify' when 'current_num_bytes' is equal to 'total_num_bytes'?  Though this all still looks shaky.  But I guess memory leak is at least better than segmentation fault.
Comment 10 Paul Pogonyshev 2008-08-09 13:18:11 UTC
Wait, or are g_file_copy() and friends synchronous?  Then we should just free 'notify' after they return and not in file_progress_callback_marshal().
Comment 11 Paul Pogonyshev 2008-08-09 13:47:58 UTC
Created attachment 116231 [details] [review]
fix?

This patch fixes it for me.

* Free 'notify' after _synchronous_ calls to g_file_copy and g_file_move.

* Use PyObject_CallFunction, I dunno what PyEval_CallFunction even is.

* Use "K", not "k": the latter doesn't work on 32-bit platforms.  Some more tweaking may be needed here, because Python docs say "K" is not available everywhere.  Also, is 'goffset' always 64-bit?


Also, this is a little fixed test script:
#! /usr/bin/env python

import gio

s = gio.File("a.txt")
d = gio.File("b.txt")

canc = gio.Cancellable()

def progress(current, total, cancellable):
    print "%2.1f%%" % (100*current/total)

ok = s.copy(
        destination=d,
        flags=gio.FILE_COPY_OVERWRITE,
        cancellable=canc,
        progress_callback=progress,
        user_data=canc
        )

print "OK? %s" % ok
Comment 12 Johan (not receiving bugmail) Dahlin 2008-08-09 14:01:15 UTC
Comment on attachment 116231 [details] [review]
fix?

Thanks!
Right, that should work.
For the asynchronous case (copy_async) we need to free the data in the callback.
I think that testcase should be incorporated in the testsuite.
Comment 13 Paul Pogonyshev 2008-08-09 15:02:44 UTC
Added a test, otherwise the patch is the same.

Sending        ChangeLog
Sending        gio/gfile.override
Sending        gio/gio.override
Sending        tests/test_gio.py
Transmitting file data ....
Committed revision 932.

2008-08-09  Paul Pogonyshev  <pogonyshev@gmx.net>

	Bug 546591 – File.copy progress_callback does not work

	* gio/gfile.override (file_progress_callback_marshal): Use
	PyObject_CallFunction() instead of PyEval_CallFunction().  Use "K"
	instead of "k" (the latter is not correct for 32-bit platforms).
	Don't free 'notify' here.
	(_wrap_g_file_copy): Free 'notify'.
	(_wrap_g_file_move): Likewise.

	* gio/gio.override (pygio_free_notify): New function.
	(async_result_callback_marshal): Use it.

	* tests/test_gio.py (TestFile.test_copy_progress.progress): New
	test.