GNOME Bugzilla – Bug 338526
Detecting python: redux
Last modified: 2006-06-08 13:29:10 UTC
Some GARNOME users have encountered problems building libxml2 because of python detection. These users typically are running older linux distos and have installed a newer version of python in some location other than /usr, for example, /opt/python-2.4.3. libxml2 is to be built using this newer version of python. The libxml2 python site_packages files are to be installed in the GARNOME tree, not the python tree. To make this happen two issues must be address: Python detection ----------------- Python should be detected in this order: * user specifies path to python executable as a configuration argument * user specifies path to python executable as an environment variable * configure script hunts in the usual places Python site_package installation -------------------------------- Python site_packages should be installed in $exec_prefix/lib/pythonX.Y/site-packages Note: in general: $exec_prefix != $python_exec_prefix Attached is a proposed patch for libxml2-2.6.23 that will resolve these issues. -Joseph
Created attachment 63497 [details] [review] patch for python path detection
+dnl a macro to check for ability to create python extensions +dnl AM_CHECK_PYTHON_HEADERS([ACTION-IF-POSSIBLE], [ACTION-IF-NOT-POSSIBLE]) +dnl function also defines PYTHON_INCLUDES +AC_DEFUN([AM_CHECK_PYTHON_HEADERS], What is this ? Why is this needed ? +dnl go looking for another Python executable +dnl this macro honors "PYTHON" if already defined + AM_PATH_PYTHON() Add a dependancy on an automake feature which I think I can't add that's why the existing detection code does not use it. I'm pretty sure I already said that previously, last year I guess. Please respect the existing code. Also you can't just change an existing configure flag anc change it's semantic like you are doing in -[ --with-python[[=DIR]] build Python bindings if found]) +[ --with-python=path build Python bindings if found]) I'm sorry but right now it seems the change opens up to far more portability and feedback problems than what it is supposed to solve. Please provide a patch based on the existing code and semantic of the configure flags, thanks, Daniel
Daniel, Sorry about the delay in getting back to you. Work sometimes gets in the way of important things like open source [:-)]... I will look again at your orginal code to see if I can rework it to make python detection and python site-path installation work as it should. Like you, I thought we had resolved these issues a year ago. I am no longer running an older distro that required a separate installation of python so I have not tracked any changes that have been made to gamin, libxml2, and libxslt. The issue resurfaced with a GARNOME user running FC3. I agree that using standard constructs that are portable is the desired endpoint. I looked at number of different implementations of python detection before choosing the one I posted. Happiness would be a standard implementation that is part of the aclocal package. The current python.m4 file, that is part of the aclocal package, does not quite do the job but it is a starting point. -Joseph
Created attachment 64108 [details] [review] patch for python path detection Daniel, Attached is a patch for configure.in that (1) works within existing code and semantics of configure flags, and (2) resolves the issues of python detection and site-package installation referenced above. Note that in addition, some quoting foo has been cleaned up within the python section. Hindsight is an exact science... If the python section were to be rewritten from scratch, using the stock macro definitions provided by aclocal in python.m4 would be a good choice. -Joseph
Okay, that looks way better :-) Applied and commited. The patch also applied for libxslt so that should fix #338527 too, thanks ! Daniel
Daniel, You are welcome. That's good news. I also patched gamin [bug #338524]. Please take a look. -Joseph
Daniel, There is a python-detection problem with 2.6.24. The python extension fails to build: [ignore line wrap] make[4]: Entering directory `/usr/local/src/GARNOME/2.14/garnome-2.14.x/src/platform/libxml2/work/main.d/libxml2-2.6.24/python' /bin/sh ../libtool --tag=CC --mode=link ccache gcc -I/usr/local/src/GARNOME/2.14/garnome-2.14.x/include -L/usr/local/src/GARNOME/2.14/garnome-2.14.x/lib -O2 -pipe -pedantic -W -Wformat -Wunused -Wimplicit -Wreturn-type -Wswitch -Wcomment -Wtrigraphs -Wformat -Wchar-subscripts -Wuninitialized -Wparentheses -Wshadow -Wpointer-arith -Wcast-align -Wwrite-strings -Waggregate-return -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Winline -Wredundant-decls -Wl,--export-dynamic -L/usr/local/src/GARNOME/2.14/garnome-2.14.x/lib -o libxml2mod.la -rpath -module -avoid-version -L../.libs libxml.lo types.lo libxml2-py.lo ../libxml2.la libtool: link: only absolute run-paths are allowed make[4]: *** [libxml2mod.la] Error 1 The source of the compile problem is pythondir = as can be seen from examining the Makefiles. How did this come about? An examination of the configure log offers a clue: Found python in environment PYTHON=/usr/bin/python Found Python version 2.4 ./configure: line 26961: PYTHON_SITE_PACKAGES: command not found Hmmm.... What's going on??? Let's look at the code in configure: if test "$with_python" != "" then pythondir=$(PYTHON_SITE_PACKAGES) else pythondir='$(libdir)/python$(PYTHON_VERSION)/site-packages' fi That looks harmless enough. The problem is upstream where PYTHON_SITE_PACKAGES is set. What's the problem there? The problem is the use of single quotes, which preserve the literal value of each character within a quoted string. I removed these to resolve this problem: --- configure- 2006-04-28 14:38:19.000000000 -0400 +++ configure 2006-04-28 14:48:49.000000000 -0400 @@ -26935,17 +26935,17 @@ -d $with_python/lib/python$PYTHON_VERSION/site-packages then PYTHON_INCLUDES=$with_python/include/python$PYTHON_VERSION - PYTHON_SITE_PACKAGES='$(libdir)/lib/python$(PYTHON_VERSION)/site-packages' + PYTHON_SITE_PACKAGES=$libdir/lib/python$PYTHON_VERSION/site-packages else if test -r $prefix/include/python$PYTHON_VERSION/Python.h then PYTHON_INCLUDES=$prefix/include/python$PYTHON_VERSION - PYTHON_SITE_PACKAGES='$(libdir)/python$(PYTHON_VERSION)/site-packages' + PYTHON_SITE_PACKAGES=$libdir/python$PYTHON_VERSION/site-packages else if test -r /usr/include/python$PYTHON_VERSION/Python.h then PYTHON_INCLUDES=/usr/include/python$PYTHON_VERSION - PYTHON_SITE_PACKAGES='$(libdir)/python$(PYTHON_VERSION)/site-packages' + PYTHON_SITE_PACKAGES=$libdir/python$PYTHON_VERSION/site-packages else echo could not find python$PYTHON_VERSION/Python.h fi @@ -26958,9 +26958,9 @@ fi if test "$with_python" != "" then - pythondir=$(PYTHON_SITE_PACKAGES) + pythondir=$PYTHON_SITE_PACKAGES else - pythondir='$(libdir)/python$(PYTHON_VERSION)/site-packages' + pythondir=$libdir/python$PYTHON_VERSION/site-packages fi else PYTHON= With these changes, configure sets PYTHON_SITE_PATH correctly, pythondir is no longer empty, and the python extension builds and installs. -Joseph
The single quotes are *on purpose*, to not break rpm build ! For RPM build you configure with a /usr prefix and run make install with different set of prefixes, if you do the evaluation at configure time and not at make install time i.e. postponed to be interpreted in the Makefile and not in configure, then this will break my releases, like it broke on Friday. The changes you suggest is *precisely* what I had to revert at the very last moment when building my releases. So, no, I won't break the rpm build, sorry ! Find another way ! You must allow evaluation to be postponed to make install time. If "make rpm" fails I won't commit any of your changes now, I must be clear about this. Daniel
Daniel, I haven't looked at the 'make rpm' issue, but will do so when I get some free cycles. Needless to say, the latest release of libxslt has the same set of problems. -Joseph
Or rather I had to apply the same set of fixes, depending from whom viewpoint you use :-) Daniel
Daniel, I believe I now have patches for gamin, libxml2, and libxslt that allow 'make rpm' to work and properly handle python-path detection. Shown below is a patch for libxml2-2.6.24. The patches for libxslt-1.1.16 and gamin-0.1.7 are similar. [I will email patches]. -Joseph =============================================================================== --- libxml2-2.6.24/configure- 2006-05-02 13:54:05.000000000 -0400 +++ libxml2-2.6.24/configure 2006-05-02 13:55:12.000000000 -0400 @@ -26935,17 +26935,17 @@ -d $with_python/lib/python$PYTHON_VERSION/site-packages then PYTHON_INCLUDES=$with_python/include/python$PYTHON_VERSION - PYTHON_SITE_PACKAGES='$(libdir)/lib/python$(PYTHON_VERSION)/site-packages' + PYTHON_SITE_PACKAGES=$libdir/python$PYTHON_VERSION/site-packages else if test -r $prefix/include/python$PYTHON_VERSION/Python.h then PYTHON_INCLUDES=$prefix/include/python$PYTHON_VERSION - PYTHON_SITE_PACKAGES='$(libdir)/python$(PYTHON_VERSION)/site-packages' + PYTHON_SITE_PACKAGES=$libdir/python$PYTHON_VERSION/site-packages else if test -r /usr/include/python$PYTHON_VERSION/Python.h then PYTHON_INCLUDES=/usr/include/python$PYTHON_VERSION - PYTHON_SITE_PACKAGES='$(libdir)/python$(PYTHON_VERSION)/site-packages' + PYTHON_SITE_PACKAGES=$libdir/python$PYTHON_VERSION/site-packages else echo could not find python$PYTHON_VERSION/Python.h fi @@ -26958,7 +26958,7 @@ fi if test "$with_python" != "" then - pythondir=$(PYTHON_SITE_PACKAGES) + pythondir='$(PYTHON_SITE_PACKAGES)' else pythondir='$(libdir)/python$(PYTHON_VERSION)/site-packages' fi