[Python-Dev] Handling linker scripts reached when dynamically loading a module

[Guido van Rossum]

Odd. Let's see what other core devs say.

On Sat, Sep 10, 2011 at 2:50 PM, Howard B. Golden
<howard_b_golden at yahoo.com> wrote:
> I don't know why, but some Linux distributions place scripts into .so
> files instead of the actual binaries. This takes advantage of a feature
> of GNU ld that it will process the script (which points to the actual
> binary) when it links the .so file.
>
> This feature works fine when you are linking a binary, but it doesn't
> take into account that binaries can be loaded dynamically by
> interpreters (e.g., Python or GHCi). If dlopen finds a linker script, it
> doesn't know what to do with it. It simply diagnoses the file as either
> an invalid ELF header or too short.
>
> On Gentoo Linux, some common libraries that are represented as linker
> scripts include libm.so, libpthread.so and libpcre.so. I know this also
> affects Ubuntu.
>
> Howard
>
> On Sat, 2011-09-10 at 14:39 -0700, Guido van Rossum wrote:
>> Excuse me for asking a newbie question, but what are linker scripts
>> and why are they important? I don't recall anyone ever having
>> requested this feature before.
>>
>> --Guido
>>
>> On Wed, Sep 7, 2011 at 12:33 PM, Howard B. Golden
>> <howard_b_golden at yahoo.com> wrote:
>> > Hi,
>> >
>> > In Haskell I experienced a situation where dynamically loaded modules
>> > were experiencing "invalid ELF header" errors. This was caused by the
>> > module names actually referring to linker scripts rather than ELF
>> > binaries. I patched the GHC runtime system to deal with these scripts.
>> >
>> > I noticed that this same patch has been ported to Ruby and Node.js, so I
>> > suggested to the libc developers that they might wish to incorporate the
>> > patch into their library, making it available to all languages. They
>> > rejected this suggestion, so I am making the suggestion to the Python
>> > devs in case it is of interest to you.
>> >
>> > Basically, when a linker script is loaded by dlopen, an "invalid ELF
>> > header" error occurs. The patch checks to see if the file is a linker
>> > script. If so, it finds the name of the real ELF binary with a regular
>> > expression and tries to dlopen it. If successful, processing proceeds.
>> > Otherwise, the original "invalid ELF error" message is returned.
>> >
>> > If you want to add this code to Python, you can look at my original
>> > patch (http://hackage.haskell.org/trac/ghc/ticket/2615) or the Ruby
>> > version (https://github.com/ffi/ffi/pull/117) or the Node.js version
>> > (https://github.com/rbranson/node-ffi/pull/5) to help port it.
>> >
>> > Note that the GHC version in GHC 7.2.1 has been enhanced to also handle
>> > another possible error when the linker script is too short, so you might
>> > also want to add this enhancement also (see
>> > https://github.com/ghc/blob/master/rts/Linker.c line 1191 for the
>> > revised regular expression):
>> >
>> > "(([^ \t()])+\\.so([^ \t:()])*):([ \t])*(invalid ELF header|file too
>> > short)"
>> >
>> > At this point, I don't have the free time to write the Python patch
>> > myself, so I apologize in advance for not providing it to you.
>> >
>> > HTH,
>> >
>> > Howard B. Golden
>> > Northridge, California, USA
>> >
>> > _______________________________________________
>> > Python-Dev mailing list
>> > Python-Dev at python.org
>> > http://mail.python.org/mailman/listinfo/python-dev
>> > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >
>>
>>
>>
>
>
>



-- 
--Guido van Rossum (python.org/~guido)