python 2.7.x on MacOSX: failed dlopen() on .so's

[Ned Deily]

On Nov 13, 2013, at 14:59 , Paul Smith <paul at mad-scientist.net> wrote:

> Thanks for the response Ned!
> 
> On Wed, 2013-11-13 at 14:40 -0800, Ned Deily wrote:
>> There shouldn't be any problems with what you are trying to do.  It
>> works for me with Python 2.7.6 and pycrypto-2.6.1.  Some suggestions:
>> - Avoid --enable-shared on OS X at least initially.  There are too
>> many ways things can go wrong.  If you've built with it, suggest
>> starting with a fresh Python source directory just to be sure.
> 
> I've tried it with all three options: no flag, --disable-shared, and
> --enable-shared, and don't notice any difference in the results; even
> with --enable-shared I don't seem to get any shared libs created.
> 
>> - Check the dynamic library dependencies of _struct.  On OS X:
>> 
>>  otool -L /Users/build/python/lib/python2.7/lib-dynload/_struct.so
> 
> I get a libgcc_s reference as well, since I'm building with GCC:
> 
> /Users/build/python/lib/python2.7/lib-dynload/_struct.so:
>        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 159.1.0)
>        /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current
> version 1105.0.0)
> 
> I wonder if using the GCC linker etc. is part of the problem?

That shouldn't be an issue.  What you do don't want to see is references to python libraries.  BTW, Xcode 4.1 is quite old and outdated for OS X 10.7.  Suggest you update to the current Xcode 4.6.3 or Command Line Tools for 10.7.

> The reason I've set PYTHONHOME is ultimately I need this installation to
> be relocatable.  It's going to be shared across lots of different
> systems and they'll have the ability to copy it wherever they want.

That could be problematic. You need to be *really* careful about how you do that.  You stand a chance with a non-shared installation.  You still should not need to set PYTHONHOME.  Also, be aware that executables and libraries built on one version of OS X are not guaranteed to work on other versions, particularly older versions unless you take certain precautions.  Even non-shared Pythons on OS X dynamically link with system-supplied libraries which can vary across os releases.  And not all libraries are supplied, so, depending on your needs, you may need to supply some additional third-party libraries.

If you need to support various OS X releases, the safest approach is to build on the oldest release to be supported, say, 10.6.  The resultant executables and libraries should then work on 10.7 through 10.9 - except every once in a while there will be a gotcha like the incompatible change in libedit for 10.9.  (Python 2.7.6 can deal with that problem.)  There is also the question if all the systems you will need to support are of the same architecture.  10.6 supports both 32-bit-only and 64-bit Intel machines.  10.5 supports in addition PPC machines.  The solution to that is to build OS X universal binaries.  If you really are careful and lucky, it is in theory possible to build on a newer system and use an OS X SDK and set MACOSX_DEPLOYMENT_TARGET for an older version and it will be downward compatible. But thorough testing is called for in that case since it is easy to have a hidden dependency.  For the python.org OS X binary installers, we go to a fair amount of trouble to build Pythons that will work across a range of OS X releases.  You might want to consider using one of them as a base.  It's usually a lot less work than trying to make it work yourself.

> 
>> - Check your other environment variables and make sure you are not
>> setting any DYLD_ or LD_ env variables.
> Hm; I am setting LD_LIBRARY_PATH to find the Python .so files.  Does
> python figure out where to look for them by itself?

Yes

--
  Ned Deily
  nad at acm.org -- []