[Cython] array expressions

[Dag Sverre Seljebotn]

On 10/14/2012 10:18 AM, Dag Sverre Seljebotn wrote:
> On 10/14/2012 08:18 AM, Stefan Behnel wrote:
>> mark florisson, 13.10.2012 20:30:
>>> On 12 October 2012 20:01, Dag Sverre Seljebotn wrote:
>>>> On 10/12/2012 05:50 PM, Robert Bradshaw wrote:
>>>>> On Fri, Oct 12, 2012 at 3:14 AM, mark florisson wrote:
>>>>>> On 12 October 2012 08:36, Stefan Behnel wrote:
>>>>>>> mark florisson, 24.08.2012 20:40:
>>>>>>>> Here a pull request for element-wise array expressions for Cython:
>>>>>>>> https://github.com/cython/cython/pull/144
>>>>>>>
>>>>>>> Mark, any news on this? I'd like to see a version merged before
>>>>>>> the master branch starts diverging all too far - it already
>>>>>>> requires a bit of adaptation.
>>>>>>> Did you manage to split off a separate minivect package?
>>>>>
>>>>> I'm assuming this has already been looked at, at least to some level,
>>>>> by Dag, but I'll try to take a brief pass at it too (probably more the
>>>>> interface than the implementation).
>>>>
>>>> Thanks for doing that, it'd be great to get this in (but myself I've
>>>> got
>>>> nothing to spare). I'll admit I was mostly focused on the generated
>>>> code and
>>>> the algorithms in minivect rather than the integration with Cython.
>>>>
>>>>> I don't see a reason for a new pull request.
>>>
>>> Great. As for the packaging, I'm creating a distribution branch, and a
>>> subtree branch. Newer versions of git have a 'subtree' command
>>> (previously https://github.com/apenwarr/git-subtree), which allows one
>>> to split of, merge, push, and pull subdirectories.
>>>
>>> This means when users pull the master project, they get the
>>> sub-projects as well (without themselves needing newer git versions).
>>> Any changes to a subproject can be merged into the subproject, ands.
>>> changes can be pulled back in (with a squash option to avoid mixing in
>>> the subproject's history).
>>>
>>> What about using this approach? That way Cython remains stable and
>>> pinned on the right minivect version now and in the future, with no
>>> burden on users.
>>
>> I still prefer having separate packages. I mean, we don't ship NumPy
>> either, even though a lot of people use Cython together with it.
>
> This is a very bad comparison. NumPy is not used by Cython at all, but
> by Cython-generated modules! Whereas minivect is a tool used in the
> compiler itself and working on the AST level.
>
> Plex would be a better comparison (though bad as well, since Plex is not
> optional while minivect is).
>
>> Keeping the two packages separate helps in keeping the interface between
>> both clean. I wouldn't want to end up with Cython shipping some
>> patched up
>> version of minivect just because it's so easy, and I would like to allow
>> users to install a new version of either Cython or minivect at any time.
>
> I think this goal (allowing separate upgrades of Cython and/or minivect)
> is unrealistic and pointless.
>
> I think you should look at minivect as "some AST transform algorithms
> which numba and Cython are able to share". It doesn't really have a life
> on its own, it's just a means for Cython and numba to cooperate. (Really
> long-term then hopefully NumPy, numexpr, Theano etc. would jump on too,
> but that won't happen just yet. If it does, we can revisit this.)

Also, I don't even know when/if numba will use it... Mark, are you able 
to fill us in on that?

Dag Sverre