GNOME Bugzilla – Bug 599991
fails to load libpython
Last modified: 2010-10-28 13:20:17 UTC
Created attachment 146482 [details] [review] fix python detection This is on Mandriva Cooker x86_64 with gst-python 0.10.17 and totem: $ totem --gst-debug-level=2 0:00:00.048235728 21041 0x1dd10e0 WARN pyplugin gstpythonplugin.c:343:plugin_init: Couldn't g_module_open libpython. Reason: /usr/lib/libpython2.6.so: cannot open shared object file: No such file or directory 0:00:00.048360036 21041 0x1dd10e0 WARN GST_PLUGIN_LOADING gstplugin.c:422:gst_plugin_register_func: plugin "/usr/lib64/gstreamer-0.10/libgstpython.so" failed to initialise Two problems here: libgstpython looks for the wrong name, libpython2.6.so is in the devel package, so it is not available on user systems, only people who build python stuff have it. It should open libpython2.6.so.1.0 instead. It should look for it in /usr/lib64 instead of /usr/lib.
First of all /usr/lib64/gstreamer-0.10/libgstpython.so should already be linked to libpythonX.Y.so.Z, that call to load libpython by hand later should really go away. Could you show the output of ldd /usr/lib64/gstreamer-0.10/libgstpython.so ?
Yes, it is linked with libpython: $ ldd /usr/lib64/gstreamer-0.10/libgstpython.so libgstreamer-0.10.so.0 => /usr/lib64/libgstreamer-0.10.so.0 (0x00002b2071a7b000) libgobject-2.0.so.0 => /usr/lib64/libgobject-2.0.so.0 (0x00002b2071d54000) libgmodule-2.0.so.0 => /usr/lib64/libgmodule-2.0.so.0 (0x00002b2071f98000) libglib-2.0.so.0 => /usr/lib64/libglib-2.0.so.0 (0x00002b207219d000) libpython2.6.so.1.0 => /usr/lib64/libpython2.6.so.1.0 (0x00002b2072462000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b2072802000) libc.so.6 => /lib64/libc.so.6 (0x00002b2072a1e000) libgthread-2.0.so.0 => /usr/lib64/libgthread-2.0.so.0 (0x00002b2072d7c000) libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00002b2072f81000) libm.so.6 => /lib64/libm.so.6 (0x00002b20732cf000) librt.so.1 => /lib64/librt.so.1 (0x00002b2073551000) libdl.so.2 => /lib64/libdl.so.2 (0x00002b2073759000) libpcre.so.0 => /lib64/libpcre.so.0 (0x00002b207395e000) libutil.so.1 => /lib64/libutil.so.1 (0x00002b2073b8c000) /lib64/ld-linux-x86-64.so.2 (0x0000555555554000) libz.so.1 => /lib64/libz.so.1 (0x00002b2073d8f000)
Then the question is, why line gstpythonplugin.c line 332 fails: g_module_symbol (g_module_open (NULL, G_MODULE_BIND_LOCAL), "Py_None", &has_python); The attached patch is not correct though, this won't work on Windows for example, or when the so version of libpython changes. Better solution would be, to let configure check for the real path/filename of libpython and let the plugin use that then.
JFYI https://bugzilla.gnome.org/show_bug.cgi?id=602075
the check fails since Py_None is a macro nowdays, the real symbol is _Py_NoneStruct. I don't see why we need the manual check at all though. Edward?
Created attachment 168687 [details] [review] Fixes the bug by not loading libpython manually at all.
(In reply to comment #5) > the check fails since Py_None is a macro nowdays, the real symbol is > _Py_NoneStruct. > > I don't see why we need the manual check at all though. Edward? TBH... I can't remember either :(
Ok I see what the check tries to do. Ideally we shouldn't link libgstpythonplugin to -lpython2.6 directly. If you have a python app loading a plugin via libgstpythonplugin, there's no need to link to libpython since the symbols are already loaded in the interpreter. I committed a fix to check for _Py_NoneStruct instead of Py_None. Now we should fix the rest to: 1) not link libgstpythonplugin to -lpython2.6 2) figure out the correct path to libpython so that it doesn't fail when we try to load it manually
What if you are loading a Python plugin, but not from a Python app? What is the advantage of not linking libgstpythonplugin to -lpython2.6? As I understand it, GStreamer will load libgstpythonplugin only when filesystem dependencies indicate that its registry entry needs to be updated, or when an element defined as a Python plugin is requested by name.
Say we link libgstpythonplugin to -lpython2.6. If you load it from a regular C app, that's fine, it will load libpython and work correctly. But say you are working with a gst-python app and you load libgstpythonplugin. That will load libpython, resulting in two different instances of the python runtime loaded in memory - one loaded from the interpreter that is running your app, one loaded from libgstpythonplugin. That would break in unexpected ways, since the python runtime has significant global state (like the GIL). The solution is to not link libgstpythonplugin to -lpython2.6. Then, when the plugin is loaded, in its plugin_init check if the python runtime is already loaded (running from a gst-python app), else load it with g_module_open. I'm working on this, hopefully i'll have something working in the next few days.
(In reply to comment #10) > Say we link libgstpythonplugin to -lpython2.6. If you load it from a regular C > app, that's fine, it will load libpython and work correctly. > > But say you are working with a gst-python app and you load libgstpythonplugin. > That will load libpython, resulting in two different instances of the python > runtime loaded in memory - one loaded from the interpreter that is running your > app, one loaded from libgstpythonplugin. That would break in unexpected ways, > since the python runtime has significant global state (like the GIL). The following is probably a stupid question. Please bear with my ignorance of how dynamic linking works. Isn't the Python shell itself linked against libpython? The Python documentation says that Py_Initialize() "is a no-op when called for a second time".
The python interpreter links to libpython statically for performance reasons.
Created attachment 168802 [details] [review] Make gstplugin load libgstpython.so with G_MODULE_BIND_LAZY.
The patch above, which makes gst_plugin_load_file load the gst-python plugin loader with G_MODULE_BIND_LAZY, is the only missing piece of the puzzle. If people don't mind this change to core too much, it is the easiest way to get python plugins working again. If people do mind, there's an alternative option, which is to split libgstpython.so in two, say libgstpython-loader.so and libgstpython.so. libgstpython-loader.so would be a regular gst plugin that g_module_open's libgstpython.so (a .so but not a gstplugin) with G_MODULE_BIND_LAZY. Discuss!
After a brief discussion on IRC, i pushed the patch above in core. This bug should be fixed now. Can people running fedora and mandriva confirm?
*** Bug 602075 has been marked as a duplicate of this bug. ***
Works for me on Mac OS.
One idea to avoid strcmp'ing for each plugin load would be to strcmp when scanning the plugin for updating the registry and for lazy listed plugins add a lazy-flag to GstPlugin, this would be preserved in the registry cached and can be used when actually loading the plugin.
Assuming this to be OBSOLETE based on comment #15. Please reopen if this is not the case. Thanks in advance.