[Cython] Status

[John Skaller2]

> On 30 Jan 2020, at 18:00, Yury V. Zaytsev <yury at shurup.com> wrote:
> 
> On 30. Jan 2020, at 03:52, John Skaller2 <skaller at internode.on.net> wrote:
>> 
>> However the shared library extensions I generate will always be linked
>> against a shared library. My build tech does not permit flat namespaces
>> for dynamic linkage. I know you CAN get Python run time as a shared
>> library on Debian based systems, but I don’t think you can get
>> a stub Python executable. Its essential that all the global data is held
>> by the library not the executable, otherwise it won’t be shared by 
>> libraries that link to it.
> 
> Users of MPI libraries have some of the same problems - maybe you could have a look at the mpi4py - coincidentally it’s also a heavy user of Cython, so a lot of inspiration can be found in the sources.
> 
> https://mpi4py.readthedocs.io/en/stable/appendix.html#mpi-enabled-python-interpreter

Thanks. That “rebuild Python from source” is also my main worry.
Luckily, I previously found, as long as you have the runtime,
its easy to build a stub like the one shown in the above page.

The problem is prebuilt binary extensions don’t work, unless the symbol
table of the library is imported into the executable so they can be re-exported
into the extensions. Which requires a flat namespace at least for the executable.

With my system, the executable is pre-built but it has no external dependencies
except C library with dlopen (or Windows equivalent). Even if that used a flat
namespace it wouldn’t help, because the shared libs I generate always link
to their dependencies with a two level namespace, so the dependence on
Python run time cannot re-export the run time library symbols. That, after
all, is the whole point of using a two level namespace.

Of course, I can also do static links which is intrinsically using a flat namespace.
So I could change it to use a flat namespace. But sometimes progress requires
avoiding stepping back decades into the medieval world :-)

[No relation to Vasily Grigoryevich Zaytsev? Just curious ..]


—
John Skaller
skaller at internode.on.net