Shared libpython, PIC, whinery (was Re: Embedding python in .so, /libpython2.2.a not position independent (hp-ux)??

[Martin v. Loewis]

Ville Vainio <vvainio at karhu.tp.spt.fi> writes:

> Back to hp-ux. Now, I have been able to recompile python w/ -fPIC and
> it seems to work ok (test suite doesn't fare any worse than it did w/o
> -fPIC). Now I can manually extract the .o's and build a .sl of my
> own. My question is: why is libpython not built to be position
> independent in the first place? 

If you are not attempting to build a shared library, compiling the
objects with -fPIC introduces a performance penalty whose magnitude
depends on the architecture.

On x86, the cost comes primarily from reserving the EBX register as
the PIC register, especially since registers are a scarce resource on
that architecture (plus causing an EBX load cycle, which is done
through a procedure call, every time a function is entered).

On PA, registers aren't that scarce. OTOH, I believe that PA has two
models of position-independent: One that uses short relative offsets
and has limits in the maximum size of a shared object (-fpic), and one
that uses large relative offsets (-fPIC). Because of the troubles of
users getting linker errors with -fpic code, gcc 3 now stops using the
small model, making -fpic synonymous to -fPIC. In turn, PIC code uses
more complex instructions to access globals, which also causes a
slow-down.

You didn't ask why libpython should not always be built as a shared
library. I feel that it would be stupid to do so by default: many
systems have problems locating their shared libraries; you'd be
essentially opening the doors for DLL hell.

> That doesn't cause platform depence, does it?

It does: you have to find out how to instruct the compiler to generate
PIC code. On some systems, it may introduce dependencies on a specific
compiler release; e.g. on Sparc64, and some Solaris versions, gcc may
create code that can be only linked if you have a Sun ld patch
installed.

> In the minimun, there should be a hint in the Makefile
> somewhere suggesting that you could make it PIC, by uncommenting
> something or whatever. The current approach requires a bit too much
> hacking to be easily repeatable (by code maintainers etc.).

It is not intended to be easily repeatable. Embedding Python in this
specific way is a too-rare application to justify anybody spending
time into investigating the details (patches are encouraged, of
course).

People have frequently attempted to contribute linking libpython as a
shared library on Unix, which has a somewhat more general
applicability (and would cover your case as well); I think there have
been three patches so far.

Unfortunately, all of these patches had to be rejected, since they did
not meet the requirements of supporting all systems correctly and
uniformly and making a shared libpython a configuration time
option. The Debian patch is the most recent in this row; it fails
these criteria by not allowing to build a statically-linked python
executable.

Regards,
Martin