Created attachment 30155 [details]
gpython.py

Created attachment 30156 [details] [review]
remove block/unblock threads in gobjectmodule.c:handler_marshal

This seems to solve it here, can you confirm?

And if you can, try to test it a bit more thoroughly (I've never really used
it, so I'm not sure what to expect/test)

If you remove pyg_block|unblock_threads then gdk_threads_enter|leave won't ever
be called.

I think a better fix would be (though I haven't tried):
  1. Change implementation of pygdk_block_threads and pygdk_unblock_threads,
replace Py_BLOCK_THREADS and Py_UNBLOCK_THREADS calls with corresponding GIL calls;
  2. From every paired PyGILState_Ensure|Relese, pyg_block|unblock_threads out
there, remove the GIL calls because they'll already be inside
pyg_block|unblock_threads.

  This would also satisfy an additional requirement that I wanted before that no
thread operations will ever get called until the user calls gtk.threads_init(),
thus no performance penalty for single threaded apps.

(Updating summary)

Gustavo: I think, like James mentioned on irc pyg_block|unblock_threads should
go away completely.

For code that calls C->Python (eg PyObject_Call[Method|Object|Function]) we
should use GIL and only Py_BEGIN_ALLOW_THREADS macros for everything else.

Created attachment 30158 [details]
Simplified testcase

Need to figure out how to manage to quit after say 500 ms (because its
obviously a deadlock) if something doesnt happen.

If we remove pyg_block|unblock_threads, then we have to remove (or deprecate)
gtk.threads_init, and make users call gtk.gdk.threads_enter|leave according to
gtk rules.  Now that's perfectly fine with me (I prefer this, in fact), just
make sure everyone is aware and agrees with this.

Why the comment "Need to figure out how to manage to quit after say 500 ms"? 
Doesn't your test already cover this with SIGALRM?

Also please note the test only works on unix because of the signal.alarm function.

Johan, your fix works for me. I'll continue testing using gpython.py to see if
there are any other problems.

Gustavo: No, they doesn't I tried to use signals, but for some reasons a signal
does not interrupt all other threads. I should probably study threads a bit further.

You should be calling threads_enter()/threads_leave() inside your timeout
function.  The GDK thread lock is not held when timeout functions are called, so
if you are going to call a GTK function in your timeout you need to manually
acquire the lock.

Also, using UNIX async signals for timeouts is likely to cause problems ...

The following deadlocks when run in gpython.py:

zippy.stooges.moeraki.com: 295:1295$ gpython.py
Interactive GTK Shell
>>> ls=ListStore(str)
>>> ls.append(['asdf'])

pyg_block/unblock_threads is dead and replaced by BEGIN/END_ALLOW_THREAD macros
and PyGILState.

gpython.py should run unmodified.

*** Bug 153978 has been marked as a duplicate of this bug. ***

This is also a regression in 2.3.97, reopening

The bug with rc1 is that since gtk_main is being called from a thread that was
created with the threading module -- so PyGILState_GetThisThreadState() is
returning NULL.  If we assume that the GIL is held when pyg_enable_threads is
called, we could use the value of the current thread state directly. 
Unfortunately, there's probably C code that calls pyg_enable_threads directly
:(.  We could claim that all such code should first acquire the GIL.

Created attachment 32038 [details] [review]
Allows gtk_main to be called on any thread

This allows gpython to work (at least it doesn't segfault).  It assumes that
the thread calling the pyg_enable_threads C function holds the GIL.

pyg_enable_threads is called from gobject.threads_init(), thus in this situation
the GIL is held.
It can also be called from external extension modules.  For this, we can just
document this requirement, which seems perfectly reasonable.

Committed 9/28 patch and closing for now.  Will reopen if needed