Created attachment 21285 [details]
The python example

Created attachment 21286 [details]
The C version

Currently, PyORBit ignores the Python threading support and global
interpreter lock.  At the same time, it probably wouldn't handle
concurrent CORBA calls on different threads if it did.

The latest version of ORBit has some multi-threading support, which I
will have to look at in relation to this.

The reason why the above test program halts is because when pycorba_orb_run is
called the global interpret lock is held. Since pycorba_orb_run calls
CORBA_ORB_run, which never returns, the interpreter is stuck there. The problem
is you can't just:
+ Py_BEGIN_ALLOW_THREADS
  CORBA_ORB_run(self->orb, &ev);
+ Py_END_ALLOW_THREADS

Because when the process responds to an incoming method call it needs to hold
the global interpreter lock to prevent messing things up. When the event loop in
CORBA_ORB_run dispatches back to pyorbit code would it be sufficient if the
first thing that happened was a Py_BLOCK_THREADS (with _save being the thread
state from pycorba_orb_run) and then the last thing it does is
Py_UNBLOCK_THREADS. This would mean that while in the orbit event loop python
threads are free to execute. I get the impression that
pyorbit_servant_generic_skel_func is the function that gets called when the
pyorbit process handles an incoming method call. Is that right? Does this make
the least bit of sense? I've run out of time today, but I might be able to try
and test my ideas out tomorrow. Please let me know if I'm on the complete wrong
track and save me some time. Let me know if the mailing list is a better place
to discuss this (I figured this way the comments get recorded).

Created attachment 26991 [details] [review]
save and restore thread state to allow multi-threading

This patch adds a global thread state variable which can be used to save the
thread state (unblock threads) at the start of pycorba_orb_run and restore the
thread state (block_threads) at the end. The same thread state is used to
restore the thread state (block_threads) at the start of
pyorbit_servant_generic_skel_func  and save the thread state (unblock threads)
at the end. This enables other threads to execute while orbit is idling in the
event loop but ensures threading is done properly when a method is called. I've
done some basic testing which seems to indicate that this works. But there are
probably other spots where blocking/unblocking needs to be done.

Created attachment 26992 [details]
pyt.idl - a basic idl file to facilitate testing python threading

Created attachment 26993 [details]
server.py - a basic threaded python server

Created attachment 26994 [details]
client.py - a basic client to invoke a method within the server

Build the type library:

orbit-idl-2 --imodule pyt.idl
gcc -fPIC -shared -o pyt-imodule.so pyt-imodule.c `pkg-config ORBit-2.0
pyorbit-2 --cflags --libs`

Run the server in one terminal:
python server.py

Run the client in another terminal:
python client.py

Normally the server just prints lots of '.' characters. As it responds to the
method call it will print some 'X' characters. The '.' and 'X' are interspersed
because the print operation (or the sleep?) yields the cpu to another thread. So
the thread running the orb.run doesn't get exclusive access to the CPU while it
is processing the method call.

Note I couldn't get the client stuff to work as another thread in the server.
Some kind of deadlock. Not sure why though.

It is probably implement this with PyGILState instead of how the current code
does.  This does change the minimum required version of Python to 2.3, but I
don't think that matters, since it has been out for a long time now and PyGTK
depends on it also.

The problem with the existing patch is that it will cause complications with
threaded Gnome programs, which would be bad.

See also http://mail.python.org/pipermail/python-dev/2003-February/033159.html

Hmm - I guess this is mostly a python related issue - if you put the ORB into
threaded mode - in theory (at least) all the ORB calls are thread-safe; leaving
only any client/wrapper logic in need of locking.

I know how to handle threads in Python; I just wasn't sure how to do it in the
ORBit2 part.  I found this FAQ entry[1]:

"  *** How do I enable a threaded ORB ?

	Before you initialize the ORB ( or bonobo ) you need to call
  linc_set_threaded (TRUE), this will cause all the locking to be setup
  correctly. Calling this after locks have been created will not work."

On the other hand, google found me an example[2] that contains this:

        (*orb) = CORBA_ORB_init(argc_ptr, argv, "orbit-local-mt-orb", ev);

instead of the linc_set_threaded call.  I don't know which version is more
correct.  In any case, I don't trust the attached C example to be correct:

        orb = CORBA_ORB_init(&argc, argv, "orbit-local-orb", &ev);

And I'd rather use Python GIL than PyEval_SaveThread()/PyEval_RestoreThread() as
in the patch.

[1] http://orbit-resource.sourceforge.net/faq2-0.2.html
[2] http://www.gnome.org/projects/ORBit2/orbit-docs/orbit/x1223.html

I asked in the orbit list:
  http://mail.gnome.org/archives/orbit-list/2005-December/msg00000.html

I'm going to have increment the required python version to 2.4, due to the new
PyEval_ThreadsInitialized() API, which I'll want to use.  I don't want any
locking when not using threads.  But Python 2.4 has been out for years, so I'll
accept no complaints! ;-)

Should be fixed in CVS, but I haven't tested yet.

Python threaded example attached to this bug report added to pyorbit.  They seem
to work fine now.