GNOME Bugzilla – Bug 736592
Support multiple interpreter instances per process
Last modified: 2014-10-01 04:52:18 UTC
I have someone trying to use mod_wsgi with pygobject. mod_wsgi creates multiple independent interpreters via Py_NewInterpreter. I believe this will cause multiple initializations of pygobject. Is that supported? If not, we should probably add an assertion somewhere.
From the docs [1]: "When the same extension is imported by another (sub-)interpreter, a new module is initialized and filled with the contents of this copy; the extension’s init function is not called." I've never tried this with pygi but it might work as is, was there a specific problem occurring?. At first I thought there might be a problem with C globals held independently of the Python module dictionary. Specifically the "async_free_list" [2]. But I don't actually think it will be a problem because the list is only added to when a callback is completed and "scheduled" for cleanup, also I believe the GIL provides safety if multiple threads are involved. In any event it would be worthwhile to add a test for this. [1] https://docs.python.org/2/c-api/init.html#c.Py_NewInterpreter [2] https://git.gnome.org/browse/pygobject/tree/gi/pygi-closure.c?id=3.13.91#n34
Also slightly related is bug 677091.
Sadly I don't think we can do anything to fix this. The Python docs state in the "Bugs and caveats" section of the C API why [1]: "Also note that combining this functionality [sub-interpreters] with PyGILState_*() APIs is delicate, because these APIs assume a bijection between Python thread states and OS-level threads, an assumption broken by the presence of sub-interpreters. It is highly recommended that you don’t switch sub-interpreters between a pair of matching PyGILState_Ensure() and PyGILState_Release() calls." We use PyGILState_Ensure() and PyGILState_Release() throughout PyGI, so sub-interpreters can't work. I also don't see a technique for detecting multiple imports of PyGI (to raise an error) in different sub-interpreters because the C extension init is not called. [1] https://docs.python.org/2/c-api/init.html#bugs-and-caveats