[Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

[Nathaniel Smith]

On Tue, Apr 9, 2013 at 2:11 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Nathaniel Smith, 09.04.2013 15:00:
>> On 9 Apr 2013 13:50, "Stefan Behnel" wrote:
>>> Nathaniel Smith, 09.04.2013 14:25:
>>> There's also the problem of dependency hell and getting rid of old modules
>>> once they are no longer used on the user side. And also, how to get them
>>> there in the first place. Having one package overwrite the files of
>>> another during its installation is just asking for trouble.
>>
>> The system I described does not involve the addition of any new files to
>> any package.
>
> I take it then that you were envisaging a separate "cython-runtime" package
> on PyPI that Cython compiled modules would have to depend on?

No, I must have been unclear -- I envisage that each cython-compiled
module continues to contain all the code it depends on (just like
now). But, the first cython-compiled module that is imported tweaks
sys.modules *as if* there were a separate cython-runtime package, and
sticks the necessary pieces (like the memoryview TypeObject) into this
module, and later modules import it from there.

There's no additional dependencies anywhere. You have to be careful to
make sure you version the runtime module appropriately so that you
don't end up with a module compiled with cython version X using the
runtime injected by a module compiled with the incompatible cython
version Y, but this is no harder than any other versioning problem.
Just make sure to change the runtime module name whenever you break
compatibility.

-n