[PyPy-issue] [issue620] AM_CHECK_PYTHON_HEADERS fails with PyPy (pypy-config?)

[Dave Malcolm]

New submission from Dave Malcolm <dmalcolm at redhat.com>:

I'm experimenting with building the python bindings for GTK against PyPy 1.4.1

The build process for PyGTK uses GNU autoconf and automake, with this automake macro::

   AM_CHECK_PYTHON_HEADERS(,[AC_MSG_ERROR(could not find Python headers)])

and I suspect there are quite a few other extension modules that use this.

The macro expands to the following test within the generated "configure" shell script::

  { $as_echo "$as_me:${as_lineno-$LINENO}: checking for headers required to compile python extensions" >&5
  $as_echo_n "checking for headers required to compile python extensions... " >&6; }
  py_prefix=`$PYTHON -c "import sys; print sys.prefix"`
  py_exec_prefix=`$PYTHON -c "import sys; print sys.exec_prefix"`
  if test -x "$PYTHON-config"; then
  PYTHON_INCLUDES=`$PYTHON-config --includes 2>/dev/null`
  else
  PYTHON_INCLUDES="-I${py_prefix}/include/python${PYTHON_VERSION}"
  if test "$py_prefix" != "$py_exec_prefix"; then
    PYTHON_INCLUDES="$PYTHON_INCLUDES -I${py_exec_prefix}/include/python${PYTHON_VERSION}"
  fi
  fi
  ... more stuff follows, which tries to compile a simple .c file containing #include <Python.h>

Attempting::

  PYTHON=pypy ./configure

fails for my RPM packages of PyPy, and presumably for the official packages of PyPy (I have the wrong libssl.so to 
run them directly)::

  checking whether pypy version >= 2.3.5... yes
  checking for pypy version... 2.5
  checking for pypy platform... linux2
  checking for pypy script directory... ${prefix}/site-packages
  checking for pypy extension module directory... ${exec_prefix}/site-packages
  checking for headers required to compile python extensions... not found
  configure: error: could not find Python headers

In the absence of a "pypy-config" in the path, it's going to use this clause::

  PYTHON_INCLUDES="-I${py_prefix}/include/python${PYTHON_VERSION}"

fwiw, PYTHON_VERSION is obtained from running::

  $PYTHON -c "import sys; print sys.version[:3]"

which for pypy-1.4.1 is "2.5"

So I believe it's looking below %{py_prefix}/include/python2.5 for a "Python.h"

However the packaging script in pypy/tool/release/package.py creates just an
"include" subdir directly below the pypydir, and copies the relevant *.h/*.inl
files there (I do something equivalent in my Fedora rpm packages)

So I believe this is likely to break building any modules built using the GNU
autotools against PyPy.

One solution for compatibility with this automake macro may be to simply
provide a "pypy-config" executable, analagous to the "python-config"
executable.

For reference, CPython generates this script from Misc/python-config.in in its Makefile:
  http://svn.python.org/view/python/branches/release25-maint/Misc/python-config.in?view=log

with this substitution:
  sed -e "s, at EXENAME@,$(BINDIR)/python$(VERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config


So perhaps PyPy's "bin" subdirectory could gain a "pypy-config" script, perhaps directly taking CPython's version 
of this?

Taking the 2.5 version above, some of the options work correctly (this is on my Fedora RPM build of PyPy)::

  $ pypy python-config.in --help
  Usage: python-config.in [--prefix|--exec-prefix|--includes|--libs|--cflags|--ldflags|--help]

  $ pypy python-config.in --prefix
  /usr/lib64/pypy-1.4.1

  $ pypy python-config.in --includes
  -I/usr/lib64/pypy-1.4.1/include -I/usr/lib64/pypy-1.4.1/include

but others fail::

  $ pypy python-config.in --exec-prefix
  Traceback (most recent call last):
    File "app_main.py", line 53, in run_toplevel
    File "python-config.in", line 36, in <module>
      print sysconfig.EXEC_PREFIX
  AttributeError: 'module' object has no attribute 'EXEC_PREFIX'
  
  $ pypy python-config.in --libs
  Traceback (most recent call last):
    File "app_main.py", line 53, in run_toplevel
    File "python-config.in", line 46, in <module>
      libs = getvar('LIBS').split() + getvar('SYSLIBS').split()
  AttributeError: 'NoneType' object has no attribute 'split'
  
  $ pypy python-config.in --cflags
  Traceback (most recent call last):
    File "app_main.py", line 53, in run_toplevel
    File "python-config.in", line 42, in <module>
      flags.extend(getvar('CFLAGS').split())
  AttributeError: 'NoneType' object has no attribute 'split'
  
  $ pypy python-config.in --ldflags
  Traceback (most recent call last):
    File "app_main.py", line 53, in run_toplevel
    File "python-config.in", line 46, in <module>
      libs = getvar('LIBS').split() + getvar('SYSLIBS').split()
  AttributeError: 'NoneType' object has no attribute 'split'

Hopefully, providing a pypy-config script seems sane (and I can hack on this, if desired).

What's the preferred approach here (e.g. either to fix up python-config, or to provide the missing 
distutils.sysconfig variables?)


(Aside: I've seen hundreds of python extension modules, and many of them use crazy non-standard build tools e.g. 
handcoded Makefiles with buggy logic for finding header files, linker options etc.  Getting them to build against 
PyPy may throw up lots of silly "yak-shaving" issues like this, on top of the more interesting one of emulating the 
CPython extension API; having a "pypy-config" may help ameliorate some of them).

----------
effort: ???
messages: 2058
nosy: dmalcolm, pypy-issue
priority: feature
release: ???
status: unread
title: AM_CHECK_PYTHON_HEADERS fails with PyPy (pypy-config?)

_______________________________________________________
PyPy development tracker <pypy-dev-issue at codespeak.net>
<https://codespeak.net/issue/pypy-dev/issue620>
_______________________________________________________