[Distutils] Dynamic linking on Linux, Mac OS, et al

[Jim Fulton]

Phillip J. Eby wrote:
> Hi folks.  I just implemented preliminary shared library build support for 
> setuptools, but I ran into a bit of a snag.  I was operating under the 
> assumption that you could simply put shared libraries in the directory 
> alongside the extensions that use them, but in practice it turns out that 
> this only works on Windows.  On Linux I found that you could only put 
> shared libraries in a directory on LD_LIBRARY_PATH, and I suspect a similar 
> issue will be found under OS X and other Unixes.
> 
> This seems to be a bit of a problem, in that it appears to mean there's no 
> sane way to include shared libraries in an egg on any platform other than 
> Windows.  It's common practice for Python extensions built for Windows to 
> include shared libraries to avoid having to install them in the Windows 
> system directories, but for Unix-like OSes this isn't really the normal 
> practice anyway.
> 
> But, in the case where a developer wants to include a shared library as 
> part of the distribution, this remains somewhat problematic.  One can of 
> course install the library to a platform-appropriate location ($prefix/lib 
> on most platforms), but to do it from an egg is harder.  You can't simply 
> unzip the egg any more, you would have to also move the libraries to a 
> standard installation location, breaking any hope of multi-version 
> support.  (And this would have to be able to happen on the fly, when shared 
> libraries and extensions are extracted to the cache directory, if running 
> zipped.)
> 
> I don't suppose you can modify LD_LIBRARY_PATH on the fly?  Any other 
> options?  Should I just give up on supporting shared libraries on anything 
> but Windows, perhaps silently converting the shared libraries to static ones?

On many Linux (Unix?) platforms, you can pass an -rpath/-R option to the
linker at build time that causes the run-time linker to use specific
paths to load libraries.  This feature is supported by disutils through
its --rpath/-R build_ext options.  This doesn't work on Mac OS X,
but it sounds like someone else may know how to make this work.

BTW, I often have need to be able to create what we call "application
buildouts" that contain various facilities needed by a particular
application.  These facilities are mostly Python-based but often include
things like relational databases, ICU, ldap servers, etc.  We want
to be able to have buildouts for multiple applications (or multiple
application versions) on a single development machine.  Each buildout
needs to be able to control versions of things it uses (including
Python versions).  We try to minimize the dependence on system libraries,
as these are usually different on development and deployment systems
and because we may need different versions for different applications on
a development machine.  I'm just beginning to learn about eggs.
I certainly hope I'll have the same flexibility when installing eggs
that I have now when using distutils.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org