[Cython] array expressions

[Dag Sverre Seljebotn]

On 10/14/2012 08:18 AM, Stefan Behnel wrote:
> mark florisson, 13.10.2012 20:30:
>> On 12 October 2012 20:01, Dag Sverre Seljebotn wrote:
>>> On 10/12/2012 05:50 PM, Robert Bradshaw wrote:
>>>> On Fri, Oct 12, 2012 at 3:14 AM, mark florisson wrote:
>>>>> On 12 October 2012 08:36, Stefan Behnel wrote:
>>>>>> mark florisson, 24.08.2012 20:40:
>>>>>>> Here a pull request for element-wise array expressions for Cython:
>>>>>>> https://github.com/cython/cython/pull/144
>>>>>>
>>>>>> Mark, any news on this? I'd like to see a version merged before
>>>>>> the master branch starts diverging all too far - it already
>>>>>> requires a bit of adaptation.
>>>>>> Did you manage to split off a separate minivect package?
>>>>
>>>> I'm assuming this has already been looked at, at least to some level,
>>>> by Dag, but I'll try to take a brief pass at it too (probably more the
>>>> interface than the implementation).
>>>
>>> Thanks for doing that, it'd be great to get this in (but myself I've got
>>> nothing to spare). I'll admit I was mostly focused on the generated code and
>>> the algorithms in minivect rather than the integration with Cython.
>>>
>>>> I don't see a reason for a new pull request.
>>
>> Great. As for the packaging, I'm creating a distribution branch, and a
>> subtree branch. Newer versions of git have a 'subtree' command
>> (previously https://github.com/apenwarr/git-subtree), which allows one
>> to split of, merge, push, and pull subdirectories.
>>
>> This means when users pull the master project, they get the
>> sub-projects as well (without themselves needing newer git versions).
>> Any changes to a subproject can be merged into the subproject, ands.
>> changes can be pulled back in (with a squash option to avoid mixing in
>> the subproject's history).
>>
>> What about using this approach? That way Cython remains stable and
>> pinned on the right minivect version now and in the future, with no
>> burden on users.
>
> I still prefer having separate packages. I mean, we don't ship NumPy
> either, even though a lot of people use Cython together with it.

This is a very bad comparison. NumPy is not used by Cython at all, but 
by Cython-generated modules! Whereas minivect is a tool used in the 
compiler itself and working on the AST level.

Plex would be a better comparison (though bad as well, since Plex is not 
optional while minivect is).

> Keeping the two packages separate helps in keeping the interface between
> both clean. I wouldn't want to end up with Cython shipping some patched up
> version of minivect just because it's so easy, and I would like to allow
> users to install a new version of either Cython or minivect at any time.

I think this goal (allowing separate upgrades of Cython and/or minivect) 
is unrealistic and pointless.

I think you should look at minivect as "some AST transform algorithms 
which numba and Cython are able to share". It doesn't really have a life 
on its own, it's just a means for Cython and numba to cooperate. (Really 
long-term then hopefully NumPy, numexpr, Theano etc. would jump on too, 
but that won't happen just yet. If it does, we can revisit this.)


Dag Sverre