[Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

[mark florisson]

On 9 April 2013 14:35, mark florisson <markflorisson88 at gmail.com> wrote:

>
>
> On 9 April 2013 14:33, mark florisson <markflorisson88 at gmail.com> wrote:
>
>>
>>
>> On 9 April 2013 14:24, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>
>>> mark florisson, 09.04.2013 15:13:
>>> > On 9 April 2013 14:11, Stefan Behnel wrote:
>>> >> Nathaniel Smith, 09.04.2013 15:00:
>>> >>> On 9 Apr 2013 13:50, "Stefan Behnel" wrote:
>>> >>>> There's also the problem of dependency hell and getting rid of old
>>> >>>> modules
>>> >>>> once they are no longer used on the user side. And also, how to get
>>> them
>>> >>>> there in the first place. Having one package overwrite the files of
>>> >>>> another during its installation is just asking for trouble.
>>> >>>
>>> >>> The system I described does not involve the addition of any new
>>> files to
>>> >>> any package.
>>> >>
>>> >> I take it then that you were envisaging a separate "cython-runtime"
>>> package
>>> >> on PyPI that Cython compiled modules would have to depend on?
>>> >>
>>> >> As long as people install their stuff using pip, that could work for
>>> them
>>> >> mostly ok, although with the regrettable Cython user impact of having
>>> to
>>> >> set that dependency for their packages in the first place.
>>> >>
>>> >> If people want to install stuff manually, dependency hell gets close.
>>> >>
>>> >> Or did you see any other ways of getting these things installed
>>> >> automatically, with a smaller user impact?
>>> >
>>> > For reference, here's a CEP about this written last year:
>>> > http://wiki.cython.org/enhancements/libcython
>>>
>>> Ok, but that CEP excludes the rather vital problem of distribution and
>>> installation. I also fail to see a reference to the problem of how
>>> multiple
>>> modules will interact that use different Cython runtime versions. That's
>>> a
>>> substantially bigger problem once symbols start becoming externally
>>> visible.
>>>
>>> Stefan
>>>
>>> _______________________________________________
>>> cython-devel mailing list
>>> cython-devel at python.org
>>> http://mail.python.org/mailman/listinfo/cython-devel
>>>
>>
>> I didn't say it was complete :) But the way I see it is basically what
>> Nathaniel said, i.e. Cython modules dependent on the runtime import it at
>> import time. It simply imports 'cython.runtime<xxx>', which has been made
>> available by the first module to initialize the runtime (compiled with
>> --include-runtime), or otherwise must be present on the filesystem. So user
>> packages can depend on a cython-runtime-x.y package (where each x.y is a
>> different package), so pip will install all the runtime versions users need
>> (or maybe we can otherwise improve upon this scheme).
>>
>
> Symbols needn't be linked, but can be imported through capsules. Maybe
> that adds some call overhead for functions, but then we should do it only
> for bigger functions which are also not likely to be called often.
>

More favourably though, it'd be nice to a sufficiently stable runtime that
avoids the need to have multiple versions available at a time. But that may
not be practical.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20130409/d5e4514b/attachment-0001.html>