python 2.7.x on MacOSX: failed dlopen() on .so's

[Ned Deily]

In article <1384370183.3496.472.camel at pdsdesk>,
 Paul Smith <paul at mad-scientist.net> wrote:
> I'm using MacOSX 10.7.5 with xcode 4.1, containing gcc 4.2.1 / clang 2.1
> (configure seems to choose gcc).
> 
> I've tried this with both Python 2.7.5 and 2.7.6.  I get the tarball,
> unpack it, then:
> 
>   $ configure --prefix=/Users/build/python
>   $ make
>   $ make install
>   $ export PATH=/Users/build/python/bin:$PATH
>   $ export PYTHONHOME=/Users/build/python
>   $ python2.7 --version
>   Python 2.7.6
> 
> Simple scripts seem to work fine.  But then I need to install various
> packages; for example pycrypto.  When I try to build that it fails to
> build:
> 
>   $ cd src/pycrypto-2.6 && python2.7 setup.py build
>   Traceback (most recent call last):
>     File "setup.py", line 45, in <module>
>       import struct
>     File "/Users/build/python/lib/python2.7/struct.py", line 1, in <module>
>       from _struct import *
>   ImportError: 
>   dlopen(/Users/build/python/lib/python2.7/lib-dynload/_struct.so, 2): Symbol 
>   not found: _PyObject_Free
>     Referenced from: /Users/build/python/lib/python2.7/lib-dynload/_struct.so
>     Expected in: dynamic lookup
> 
> This happens with every module that is not pure-Python: boto, paramiko,
> etc.
> 
> Looking at the various python2.7/lib-dynload/*.so files with nm I see
> that actually a number of them refer to an unresolved ("U") symbol
> _PyObject_Free (and other _PyObject_* symbols).  I can't find any shared
> library being built by the python build (.dylib or .so or whatever);
> there's just a libpython2.7.a file (I tried forcing --enable-shared on
> the configure line and it didn't change anything).

There shouldn't be any problems with what you are trying to do.  It works for 
me with Python 2.7.6 and pycrypto-2.6.1.  Some suggestions:
- Avoid --enable-shared on OS X at least initially.  There are too many ways 
things can go wrong.  If you've built with it, suggest starting with a fresh 
Python source directory just to be sure.
- Check the dynamic library dependencies of _struct.  On OS X:

  otool -L /Users/build/python/lib/python2.7/lib-dynload/_struct.so

For a non-shared build, the only library dependency should be 
/usr/lib/libSystem.B.dylib.
- Make sure you are *really* building pycrypto and friends with your Python 
and not with some other one.
- On OS X with an installed Python, you shouldn't need to set PYTHONHOME.

-- 
 Ned Deily,
 nad at acm.org