[Python-Dev] Handling linker scripts reached when dynamically loading a module

[Howard B. Golden]

I don't know why, but some Linux distributions place scripts into .so
files instead of the actual binaries. This takes advantage of a feature
of GNU ld that it will process the script (which points to the actual
binary) when it links the .so file.

This feature works fine when you are linking a binary, but it doesn't
take into account that binaries can be loaded dynamically by
interpreters (e.g., Python or GHCi). If dlopen finds a linker script, it
doesn't know what to do with it. It simply diagnoses the file as either
an invalid ELF header or too short.

On Gentoo Linux, some common libraries that are represented as linker
scripts include libm.so, libpthread.so and libpcre.so. I know this also
affects Ubuntu.

Howard

On Sat, 2011-09-10 at 14:39 -0700, Guido van Rossum wrote:
> Excuse me for asking a newbie question, but what are linker scripts
> and why are they important? I don't recall anyone ever having
> requested this feature before.
> 
> --Guido
> 
> On Wed, Sep 7, 2011 at 12:33 PM, Howard B. Golden
> <howard_b_golden at yahoo.com> wrote:
> > Hi,
> >
> > In Haskell I experienced a situation where dynamically loaded modules
> > were experiencing "invalid ELF header" errors. This was caused by the
> > module names actually referring to linker scripts rather than ELF
> > binaries. I patched the GHC runtime system to deal with these scripts.
> >
> > I noticed that this same patch has been ported to Ruby and Node.js, so I
> > suggested to the libc developers that they might wish to incorporate the
> > patch into their library, making it available to all languages. They
> > rejected this suggestion, so I am making the suggestion to the Python
> > devs in case it is of interest to you.
> >
> > Basically, when a linker script is loaded by dlopen, an "invalid ELF
> > header" error occurs. The patch checks to see if the file is a linker
> > script. If so, it finds the name of the real ELF binary with a regular
> > expression and tries to dlopen it. If successful, processing proceeds.
> > Otherwise, the original "invalid ELF error" message is returned.
> >
> > If you want to add this code to Python, you can look at my original
> > patch (http://hackage.haskell.org/trac/ghc/ticket/2615) or the Ruby
> > version (https://github.com/ffi/ffi/pull/117) or the Node.js version
> > (https://github.com/rbranson/node-ffi/pull/5) to help port it.
> >
> > Note that the GHC version in GHC 7.2.1 has been enhanced to also handle
> > another possible error when the linker script is too short, so you might
> > also want to add this enhancement also (see
> > https://github.com/ghc/blob/master/rts/Linker.c line 1191 for the
> > revised regular expression):
> >
> > "(([^ \t()])+\\.so([^ \t:()])*):([ \t])*(invalid ELF header|file too
> > short)"
> >
> > At this point, I don't have the free time to write the Python patch
> > myself, so I apologize in advance for not providing it to you.
> >
> > HTH,
> >
> > Howard B. Golden
> > Northridge, California, USA
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
> >
> 
> 
>