GNOME Bugzilla – Bug 758095
Autotools updates and python handling simplification
Last modified: 2021-07-05 10:59:38 UTC
Created attachment 315445 [details] [review] Use AM_PATH_PYTHON to find python Hello, while working on backporting to security fixes for Gentoo, I took some time to try to enhance python detection in autoconf and fix a few others things. While this is not perfect, it should improve readability a lot.
Created attachment 315446 [details] [review] Disable building of python static module A patch we have be carrying for a while. I see that fedora spec also explicitly deletes the static archive.
Created attachment 315447 [details] [review] Use automake targets to install python examples Another slight enhancement to leverage automake feature rather than try to reproduce them with manual make rules.
I also started work on make libxslt support python3 but got stuck on a unittest failing to find root element. If you are interested in this anyway, I can submit a patch in another bug report. There is a lot more to do autotools wise I think, so if you are interested in this work, just say so :)
Libxml2 doesn't use AM_PATH_PYTHON. This potentially affects portability. Did you tested on all platorms supported by libxml2 ? I mean non gentoo, gygwin, mingw, all the various unixes, MacOSes, and bizarre platform out there. If no, then NACK, you may use AM_PATH_PYTHON only if there is a way to detect it worked then fallback to previous code if it didn't. But removing the old code without justification "it's enhanced" I'm sorry but that sounds very risky without a clear gain, Daniel
Now for python3 libxml2 has support, libxslt one should be similar, I had initial patches but never time for finishing it. So interested but cautious, thanks, Daniel
Well AM_PATH_PYTHON comes from the automake guys, as you can see here: http://git.savannah.gnu.org/gitweb/?p=automake.git;a=blob;f=m4/python.m4;h=32c30e29991073cb1cfc1e6034cd184e5f9782ec;hb=HEAD#l35, it does pretty much the same you had already. It is quite standard and used in packages such as gobject-introspection, libpeas, libgsf, gtk-vnc, packagekit, pygobject, pygtk, etc. I know most of these do build on multiple architectures, including OS X, not sure about cygwin and mingw to be honest but the code is pretty much the same as before. Also, I have used it on debian as well while working at SmartJog years ago, see https://github.com/SmartJog. I hope this re-assures you in this being generic work targeting at upstream using best practices and nothing targeted at my distro. As for libxml2, my plan is to provide the same patch there. For python3, I'll prepare the patch and submit it asap.
I have an OSX setup with homebrew at work so I took 5 minutes to test on it and it does work. Darwin localhost 15.0.0 Darwin Kernel Version 15.0.0: Wed Aug 26 16:57:32 PDT 2015; root:xnu-3247.1.106~1/RELEASE_X86_64 x86_64 Only problem I had was that I had to install my libxml2 and libxslt clones to have unittests pass, probably because of OSX provided libxml2/libxslt shadowing (difference in thread being enabled or not I suspect) the ones from sources when the python interpreter tries to load them. I haven't dig it too much but the error was reproducible with v1.1.28 tag and master without my patches applied. With current autoconf code, I had to introduce a little workaround for it to set PYTHON_INCLUDES though but maybe that's because I couldn't find the proper way to do it. diff --git a/configure.in b/configure.in index e84ad4f..f9109e8 100644 --- a/configure.in +++ b/configure.in @@ -353,6 +353,7 @@ except: print 0"` fi if test ! -d "$PYTHON_SITE_PACKAGES" then + PYTHON_INCLUDES=`$PYTHON$PYTHON_VERSION-config --includes` PYTHON_SITE_PACKAGES=`$PYTHON -c "from distutils import sysconfig; print sysconfig.get_python_lib()"` fi fi note that I used $PYTHON for python-config because --with-python provided PATH might not match the one in env leading to a different python-config being found.
Created attachment 316150 [details] [review] Use AM_PATH_PYTHON to find python Updated patch that avoids relying on environment to confirm python support is wanted.
I am against disabling static build along with shared as libxml2 and libxslt are used in a *lot* of embedded systems and those often rely on static. If I count the full user base of Genttoo and Fedora and compare to embedded use I'm pretty certain the embbeded consistute the very vast majority of the deploymenets, so please stop considering changes just from the narrow view point of the distros. Thanks, Daniel
Did you consider https://bugzilla.gnome.org/show_bug.cgi?id=685603#c3 ? I could easily update the patch to automatically disable python support when you want to build static libraries.
I don't see the relation between disabling/enabling static builds and enabling/disabling python, this is completely orthogonal and should remain so. Daniel
Python never imports code from *.a files. The list of supported extensions is in _PyImport_DynLoadFiletab in Python/dynload_shlib.c: https://github.com/python/cpython/blob/2.7/Python/dynload_shlib.c#L34 https://github.com/python/cpython/blob/3.3/Python/dynload_shlib.c#L33 https://github.com/python/cpython/blob/3.4/Python/dynload_shlib.c#L29 https://github.com/python/cpython/blob/3.5/Python/dynload_shlib.c#L29 https://github.com/python/cpython/blob/3.6/Python/dynload_shlib.c#L29 https://github.com/python/cpython/blob/master/Python/dynload_shlib.c#L30 In Python >=3.3, this list can be seen at run-time using: pythonX.Y -c 'import importlib.machinery; print(importlib.machinery.EXTENSION_SUFFIXES)' Building of *.a files for Python is a waste of time. Installing them is a waste of disk space.
> Python never imports code from *.a files. evidently, but you know what, very large majority of libxml2 users especially embedded don't use python bindings, and some need .a . and there is one configure file for both. I don't see why building libxml2.a should in any way prevent also building the python bindings. So explain your position because right now that's just irrelevant and I will give my users priority, not random people on this bug. Clear ? Daniel
(This bug is about libxslt, not libxml2.) (In reply to Daniel Veillard from comment #9) > I am against disabling static build along with shared Patch from comment #1 only disables building of libxsltmod.a in python subdirectory. It does not disable building of libxslt.a in libxslt subdirectory or libexslt.a in libexslt subdirectory.
Let's try to put things another way: * we do not want to break any existing usage * python does not make use of static librairies * we suggest passing adequate flags to libtool for the python module only now you said it causes problems for embedded people that build things static, then: * either static and dynamic support is enabled and there is no problem * only static support is enabled and python bindings build will fail If I'm not mistaken up to this point, our suggestion is to disable building the python binding only in this last case to avoid preventable build failures and add a warning in configure output if you deem it necessary. Or we assume embedded people know better and just let them configure as they please.
Right if that's only for python it makes sense. I have had some issues with Fedora trying to get rid of the main static libs that's the context of my feedback. For python, sure ... Daniel
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org. As part of that, we are mass-closing older open tickets in bugzilla.gnome.org which have not seen updates for a longer time (resources are unfortunately quite limited so not every ticket can get handled). If you can still reproduce the situation described in this ticket in a recent and supported software version, then please follow https://wiki.gnome.org/GettingInTouch/BugReportingGuidelines and create a new ticket at https://gitlab.gnome.org/GNOME/libxslt/-/issues/ Thank you for your understanding and your help.