distutils, PyBison and Mac OS X

[Maarten Sneep]

I'm trying to build PyBison[1] on Mac OS X, and I'm running into some
problems with the distutils.

Just for starters: PyBison requires Pyrex[2]. This is not a problem,
and Pyrex seems to work without problems, at least the primes sample
module shows a nice 25 to 30 fold speed increase over the pure python
version.

I used the distutils to create the module from the primes.pyx sample,
following the setup.py from the PyBison distrubution:

[code]
from distutils.core import setup, Extension
from Pyrex.Distutils import build_ext

setup( name = "primes", version = "1.0",
  description="Pyrex sample for calculating prime numbers",
  author='Greg Ewing', 
  url='http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/',
  ext_modules=[Extension("primes", ["primes.pyx"])],
  cmdclass = {'build_ext': build_ext}
)
[/code]

I mention this, because PyBison seems to use the distutils to compile
the bison and flex source code on the fly, but fails to pick up some
parameters, which prevents it from working correctly.

Although Mac OS X uses gcc, and is an otherwise pretty complete unix
install, it is missing some components: shared libraries are not
linked with -shared but use -bundle instead. A succesful build of
primes is linked with:

export MACOSX_DEPLOYMENT_TARGET=10.3; gcc -Wl,-F. -Wl,-F. \
    -bundle -undefined dynamic_lookup \
    build/temp.darwin-7.9.0-Power_Macintosh-2.3/primes.o \
    -o primes.so

(Another difference that caused major headaches: the absense of a
'dl' library has been resolved with version 10.3 ("Panther"). A dl
library is included as a wrapper around the bundles mechanism that
were present all along. Since the linux version uses the dl library,
I tried to compile the same source, and that part seems to run just
fine).

The issues: 

- distutils.ccompiler uses the unixcompiler, which has hard coded a
  cc -shared to create shared objects. I've tried to override this,
  but no luck there (other issues seem to crop up). Is this a known
  issue with distutils?
  
  I installed pybison using the linux C-source, and tried it out
  using the 'calc' example. After setting verbose=1 in the run.py,
  and enabling the "int lineno = 0" in calc.py, the compilation phase
  finishes without a hitch, but linking fails badly:
  
    bisonCmd=['bison', '-d', '-v', '-t', 'tmp.y']
    renaming bison output files
    tmp.tab.c => tmp.bison.c
    tmp.tab.h => tokens.h
    cc: unrecognized option `-shared'
    ld: Undefined symbols:
    _main
    _PyInt_FromLong
    _PyObject_GetAttrString
    _PyObject_HasAttrString
    _PyObject_SetAttrString
    _PyString_AsString
    _PyString_FromString
    _PyTuple_New
    _PyTuple_SetItem
    __Py_NoneStruct
    Traceback (most recent call last):
      File "run.py", line 7, in ?
        parser = calc.Parser(verbose=1, keepfiles=1)
      File ". . ./python2.3/site-packages/bison.py", line 308, 
        in __init__ self.engine = ParserEngine(self)
      File "bison_.pyx", line 187, in bison_.ParserEngine.__init__
      File "bison_.pyx", line 202, in 
        bison_.ParserEngine.openCurrentLib
      File "bison_.pyx", line 540, in bison_.ParserEngine.buildLib
      File ". . ./python2.3/distutils/ccompiler.py", line 843, 
        in link_shared_object extra_preargs, extra_postargs, 
        build_temp, target_lang)
      File ". . ./python2.3/distutils/unixccompiler.py", line 178, 
        in link raise LinkError, msg
    distutils.errors.LinkError: command 'cc' failed with exit status 1
  
- Would it be possible to modify PyBison to use a call to the 
  distutils.core.setup(. . .) function and call that as if 'python 
  setup.py build_ext --inplace' had been called from the command line, 
  since that seems to pick up the correct options.
  
- Where do I start: distutils is pretty large, and it seems to have
  the right options, but they just aren't used in PyBison.

Thanks in advance for any insights that get me going,

Maarten

Links:
[1] http://www.freenet.org.nz/python/pybison/
[2] http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/