[C++-SIG] Re: C++ extension module crashes on Solaris

[Geoffrey Furnish]

Okay, I'm not an expert on these issues either, but let me offer some
thoughts.

First, it is absolutely necessary that shared things (shared libs,
dynloadable modules) must be compiled with position independent code.
In this case, that means that Python itself does not have to be
compiledwith PIC, but the loadable extensions modules /do/ /have/ to
be compiled with -fPIC.  ("-fPIC" is the gcc option, some other vendor 
compilers have called this "-z" and probably other variants.)

Secondly, the loadable extesnion module itself has external symbol
references that must be bound when the module is loaded.  One way to
do this, such as is often used when building a shared library (or a "dso"
on Irix, for example), is to actually link the shared lib against
other shared libs.  Your resulting libXYZ.so in such a case, will
typically have bound into it, the information that the dynamic loader
would need in order to resolve outstanding references.  In such a
case, linking an app against this .so would implicilty result in other 
.so's (the ones your .so was linked against) getting pulled into the
client app.

This case is different.  you aren't building a shared lib, and your
link line doesn't link your extension module against any shared libs.
Instead, you are expecting to pick up your missing external symbols by 
having them bound against the symbols in the enclsoing process
context.  i.e., Python itself.

In order for this to work, some systems (I don't know whether Slowaris 
is like this or not, but some are, so it might be) require that the
host application must "export symbols" so that the dynamic loader can
consider them candidates for symbol resolution when dso's are
dynloaded. 

In other words:
   link your Python with "-Wl,-E" (but it doesn't have to 
	be compiled -fPIC)
   Compile your extension module with -fPIC, lijnke with -shared.

It seems you're now doing this last step, but I don't think we've seen 
from the info presented earlier if you're doing the first step.

BTW, some compilers call this "-export-dynamic" instead of making you
pass bizarre options to the linker as in "-Wl,-E".

Hope this helps.  Of course, I might be out to lunch on this one...  :-).

Steve Harris writes:
 > "Lyle Johnson" <jlj at cfdrc.com> writes:
 > 
 > [...]
 > 
 > > I use these flags to compile the individual files:
 > > 
 > > 	g++ -fPIC -c file1.cpp
 > > 	g++ -fPIC -c file2.cpp
 > > 
 > > and then for the link step:
 > > 
 > > 	g++ -Wl,-E -shared -o seh_test.so file1.o file2.o ...
 > > 
 > > I noticed in particular that you are not using the "-fPIC" flag for
 > > the compile step. I know that this is REQUIRED on most platforms for
 > > this to work properly, so it might be the missing piece of the
 > > puzzle in your case.
 > 
 > I didn't recompile Python yet, but I did rebuild my extension module
 > and got a similar, but different result:
 > 
 > 
 > $ make
 > g++ -fPIC -I../Include -g -O2 \
 >   -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \
 >   -c ./pod.cc
 > g++ -fPIC -I../Include -g -O2 \
 >   -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \
 >   -c ./pypod.cc
 > g++ -fPIC -I../Include -g -O2 \
 >   -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \
 >   -c ./seh_test.cc
 > g++ -fPIC -I../Include -g -O2 \
 >   -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \
 >   -c ./../Src/cxx_extensions.cxx
 > gcc -I../Include -g -O2 \
 >   -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \
 >   -c ./../Src/cxxextensions.c
 > g++ -fPIC -I../Include -g -O2 \
 >   -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \
 >   -c ./../Src/cxxsupport.cxx
 > g++ -shared -Wl,-E  \
 >   pod.o pypod.o seh_test.o \
 >   cxx_extensions.o cxxextensions.o cxxsupport.o \
 >   -o seh_test.so
 > 
 > $ cp seh_test.so ~/usr/local/lib/python1.5/
 > $ python
 > Python 1.5.2 (#1, May 19 2000, 09:34:45)  [GCC 2.95.2 19991024 (release)] on sunos5
 > Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
 > >>> import seh_test
 > Traceback (innermost last):
 >   File "<stdin>", line 1, in ?
 > ImportError: ld.so.1: python: fatal: relocation error: file ./seh_test.so: symbol PyExc_RuntimeError: referenced symbol not found
 > >>>
 > 
 > 
 > Note that this time, it's the symbol PyExc_RuntimeError that's causing
 > the problem.
 > 
 > Does it matter that the gcc line above (for ../Src/cxxextensions.c)
 > doesn't have the 'fPIC' option on it?
 > 
 > -- 
 > Steven E. Harris
 > Primus Knowledge Solutions, Inc.
 > http://www.primus.com
 > 
 > _______________________________________________
 > C++-SIG maillist  -  C++-SIG at python.org
 > http://www.python.org/mailman/listinfo/c++-sig
 >