After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 736592 - Support multiple interpreter instances per process
Support multiple interpreter instances per process
Status: RESOLVED WONTFIX
Product: pygobject
Classification: Bindings
Component: introspection
unspecified
Other Linux
: Normal normal
: ---
Assigned To: Nobody's working on this now (help wanted and appreciated)
Python bindings maintainers
Depends on:
Blocks:
 
 
Reported: 2014-09-12 21:28 UTC by Colin Walters
Modified: 2014-10-01 04:52 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Colin Walters 2014-09-12 21:28:11 UTC
I have someone trying to use mod_wsgi with pygobject.  mod_wsgi creates multiple independent interpreters via Py_NewInterpreter.

I believe this will cause multiple initializations of pygobject.  Is that supported?  If not, we should probably add an assertion somewhere.
Comment 1 Simon Feltman 2014-09-13 00:59:24 UTC
From the docs [1]:

"When the same extension is imported by another (sub-)interpreter, a new module is initialized and filled with the contents of this copy; the extension’s init function is not called."

I've never tried this with pygi but it might work as is, was there a specific problem occurring?.

At first I thought there might be a problem with C globals held independently of the Python module dictionary. Specifically the "async_free_list" [2]. But I don't actually think it will be a problem because the list is only added to when a callback is completed and "scheduled" for cleanup, also I believe the GIL provides safety if multiple threads are involved.

In any event it would be worthwhile to add a test for this.

[1] https://docs.python.org/2/c-api/init.html#c.Py_NewInterpreter
[2] https://git.gnome.org/browse/pygobject/tree/gi/pygi-closure.c?id=3.13.91#n34
Comment 2 Simon Feltman 2014-09-13 01:01:51 UTC
Also slightly related is bug 677091.
Comment 3 Simon Feltman 2014-10-01 04:52:18 UTC
Sadly I don't think we can do anything to fix this. The Python docs state in the "Bugs and caveats" section of the C API why [1]:

"Also note that combining this functionality [sub-interpreters] with PyGILState_*() APIs is delicate, because these APIs assume a bijection between Python thread states and OS-level threads, an assumption broken by the presence of sub-interpreters. It is highly recommended that you don’t switch sub-interpreters between a pair of matching PyGILState_Ensure() and PyGILState_Release() calls."

We use PyGILState_Ensure() and PyGILState_Release() throughout PyGI, so sub-interpreters can't work. I also don't see a technique for detecting multiple imports of PyGI (to raise an error) in different sub-interpreters because the C extension init is not called.

[1] https://docs.python.org/2/c-api/init.html#bugs-and-caveats