[Cython] array expressions

[Dag Sverre Seljebotn]

On 10/14/2012 10:32 AM, Stefan Behnel wrote:
> Dag Sverre Seljebotn, 14.10.2012 10:18:
>> On 10/14/2012 08:18 AM, Stefan Behnel wrote:
>>> mark florisson, 13.10.2012 20:30:
>>>> On 12 October 2012 20:01, Dag Sverre Seljebotn wrote:
>>>>> On 10/12/2012 05:50 PM, Robert Bradshaw wrote:
>>>>>> On Fri, Oct 12, 2012 at 3:14 AM, mark florisson wrote:
>>>>>>> On 12 October 2012 08:36, Stefan Behnel wrote:
>>>>>>>> mark florisson, 24.08.2012 20:40:
>>>>>>>>> Here a pull request for element-wise array expressions for Cython:
>>>>>>>>> https://github.com/cython/cython/pull/144
>>>>>>>>
>>>>>>>> Mark, any news on this? I'd like to see a version merged before
>>>>>>>> the master branch starts diverging all too far - it already
>>>>>>>> requires a bit of adaptation.
>>>>>>>> Did you manage to split off a separate minivect package?
>>>>>>
>>>>>> I'm assuming this has already been looked at, at least to some level,
>>>>>> by Dag, but I'll try to take a brief pass at it too (probably more the
>>>>>> interface than the implementation).
>>>>>
>>>>> Thanks for doing that, it'd be great to get this in (but myself I've got
>>>>> nothing to spare). I'll admit I was mostly focused on the generated
>>>>> code and
>>>>> the algorithms in minivect rather than the integration with Cython.
>>>>>
>>>>>> I don't see a reason for a new pull request.
>>>>
>>>> Great. As for the packaging, I'm creating a distribution branch, and a
>>>> subtree branch. Newer versions of git have a 'subtree' command
>>>> (previously https://github.com/apenwarr/git-subtree), which allows one
>>>> to split of, merge, push, and pull subdirectories.
>>>>
>>>> This means when users pull the master project, they get the
>>>> sub-projects as well (without themselves needing newer git versions).
>>>> Any changes to a subproject can be merged into the subproject, ands.
>>>> changes can be pulled back in (with a squash option to avoid mixing in
>>>> the subproject's history).
>>>>
>>>> What about using this approach? That way Cython remains stable and
>>>> pinned on the right minivect version now and in the future, with no
>>>> burden on users.
>>>
>>> I still prefer having separate packages. I mean, we don't ship NumPy
>>> either, even though a lot of people use Cython together with it.
>>
>> This is a very bad comparison. NumPy is not used by Cython at all, but by
>> Cython-generated modules! Whereas minivect is a tool used in the compiler
>> itself and working on the AST level.
>>
>> Plex would be a better comparison (though bad as well, since Plex is not
>> optional while minivect is).
>>
>>> Keeping the two packages separate helps in keeping the interface between
>>> both clean. I wouldn't want to end up with Cython shipping some patched up
>>> version of minivect just because it's so easy, and I would like to allow
>>> users to install a new version of either Cython or minivect at any time.
>>
>> I think this goal (allowing separate upgrades of Cython and/or minivect) is
>> unrealistic and pointless.
>>
>> I think you should look at minivect as "some AST transform algorithms which
>> numba and Cython are able to share". It doesn't really have a life on its
>> own, it's just a means for Cython and numba to cooperate. (Really long-term
>> then hopefully NumPy, numexpr, Theano etc. would jump on too, but that
>> won't happen just yet. If it does, we can revisit this.)
>
> Ok, so I gave bad examples, fair enough. But I still don't see why we
> should integrate minivect with Cython more deeply than necessary. Could you
> explain what the advantage would be over having two separate packages?

Mark already raised the issue and we discussed in through in another 
thread where you participated, and me, Mark and Robert at least agreed 
on something very similar to what Mark is proposing to do now.

If you want to reopen the discussion I wish you had rather dug up the 
old thread again and respond to arguments in it, rather than us having 
to repeat the arguments here.

(Though please also consider what it does to discussion climate to 
reopen discussions about trivial details that can also trivially be 
changed later if better arguments crop up. Mark had a weekend of spare 
time to get this merged (yay!), and now who-knows-what happens because 
you decided to bikeshed something that had, at least to the eyes of rest 
of us, achieved consensus on this list.)

Dag