[Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

[mark florisson]

On 9 April 2013 14:33, mark florisson <markflorisson88 at gmail.com> wrote:

>
>
> On 9 April 2013 14:24, Stefan Behnel <stefan_ml at behnel.de> wrote:
>
>> mark florisson, 09.04.2013 15:13:
>> > On 9 April 2013 14:11, Stefan Behnel wrote:
>> >> Nathaniel Smith, 09.04.2013 15:00:
>> >>> On 9 Apr 2013 13:50, "Stefan Behnel" wrote:
>> >>>> There's also the problem of dependency hell and getting rid of old
>> >>>> modules
>> >>>> once they are no longer used on the user side. And also, how to get
>> them
>> >>>> there in the first place. Having one package overwrite the files of
>> >>>> another during its installation is just asking for trouble.
>> >>>
>> >>> The system I described does not involve the addition of any new files
>> to
>> >>> any package.
>> >>
>> >> I take it then that you were envisaging a separate "cython-runtime"
>> package
>> >> on PyPI that Cython compiled modules would have to depend on?
>> >>
>> >> As long as people install their stuff using pip, that could work for
>> them
>> >> mostly ok, although with the regrettable Cython user impact of having
>> to
>> >> set that dependency for their packages in the first place.
>> >>
>> >> If people want to install stuff manually, dependency hell gets close.
>> >>
>> >> Or did you see any other ways of getting these things installed
>> >> automatically, with a smaller user impact?
>> >
>> > For reference, here's a CEP about this written last year:
>> > http://wiki.cython.org/enhancements/libcython
>>
>> Ok, but that CEP excludes the rather vital problem of distribution and
>> installation. I also fail to see a reference to the problem of how
>> multiple
>> modules will interact that use different Cython runtime versions. That's a
>> substantially bigger problem once symbols start becoming externally
>> visible.
>>
>> Stefan
>>
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>>
>
> I didn't say it was complete :) But the way I see it is basically what
> Nathaniel said, i.e. Cython modules dependent on the runtime import it at
> import time. It simply imports 'cython.runtime<xxx>', which has been made
> available by the first module to initialize the runtime (compiled with
> --include-runtime), or otherwise must be present on the filesystem. So user
> packages can depend on a cython-runtime-x.y package (where each x.y is a
> different package), so pip will install all the runtime versions users need
> (or maybe we can otherwise improve upon this scheme).
>

Symbols needn't be linked, but can be imported through capsules. Maybe that
adds some call overhead for functions, but then we should do it only for
bigger functions which are also not likely to be called often.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20130409/3470ad10/attachment.html>