[Cython] Shared Cython runtime

[Stefan Behnel]

Stefan Behnel, 09.04.2013 17:12:
> mark florisson, 09.04.2013 16:51:
>> On 9 April 2013 15:47, Stefan Behnel wrote:
>>
>>> mark florisson, 09.04.2013 16:32:
>>>> On 9 April 2013 14:55, Nikita Nemkin wrote:
>>>>> One alternative for code reuse in large Cython projects
>>>>> could be packaging multiple modules into one shared library.
>>>>
>>>> We have 'include'! :) Seriously though, that wouldn't work well with the
>>>> import mechanism, and probably not for C compile time either.
>>>
>>> CPython has support for it. It's actually quite easy, you basically just
>>> register all module init functions explicitly at init time and let CPython
>>> do the rest.
>>>
>>> So, compiling a package into one big Cython module with shared internal
>>> types and utility code should be quite simple and would IMHO be a very cool
>>> feature.
>>
>> I was unaware of that, it'd be a nice feature then (though still not imho
>> the right solution to a runtime system). How does this work? Feel free to
>> point me at some documentation.
> 
> It's the inittab mechanism, as described starting here:
> 
> http://docs.python.org/2/c-api/import.html#PyImport_AppendInittab

Oh, and even simpler, you can just stick multiple Cython modules with their
module init functions into one big shared library and have the "main" init
function of that module call the others. That will stick them into
sys.modules as a side effect, so it's not really different from going
through Python's import machinery for each module separately (just way more
efficient - at least if you really need all of the modules...).

There may be glitches with stuff like "__file__" and friends, but that
should not be any worse than the current way of workings.

Stefan