[Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

[Stefan Behnel]

Nathaniel Smith, 09.04.2013 14:25:
> On 9 Apr 2013 13:05, "mark florisson" wrote:
>> However, the memoryview class is duplicated in every cython module, which
>> means a memoryview object from another module will fail this check. This is
>> a general problem in Cython that could be worked around for memoryviews,
>> but in general the lack of a Cython runtime is a blessing for distribution
>> purposes and a wart for most other purposes.
> 
> This seems easy to fix - a Cython runtime would be a python module like
> cython.runtime that Cython modules imported and used, right? You could
> emulate this perfectly by just checking for such a module in sys.modules at
> import time, and injecting a copy if none was found. (With some versioning
> in the name so modules compiled with different versions of Cython could
> coexist.)> This would be much easier than coming up with elaborate object
> model hacks, since it just uses existing standard python mechanisms...

"easy" may not be the optimal way of putting this - otherwise, it would
have been done long ago, back when we decided that it should eventually
work that way.

The main problems is that this would move code out of the module, and thus
out of the module's control and out of the sight of the C compiler, so we'd
need to decide rather carefully what to externalise, based on criteria like
API stability, performance, actual doability, ...

There's also the problem of dependency hell and getting rid of old modules
once they are no longer used on the user side. And also, how to get them
there in the first place. Having one package overwrite the files of another
during its installation is just asking for trouble.

"easy" seems rather the right word for the current situation, given all of
the open questions that are linked to this can of worms.

Oh, and not to forget that someone actually has to do it.

Stefan