[Cython] array expressions

[Dag Sverre Seljebotn]

On 08/27/2012 11:53 AM, Dag Sverre Seljebotn wrote:
> On 08/24/2012 08:40 PM, mark florisson wrote:
>> Hey,
>>
>> Here a pull request for element-wise array expressions for Cython:
>> https://github.com/cython/cython/pull/144
>> It includes the IndexNode refactoring branch as well.
>>
>> This has been the work this last summer for the gsoc, with great
>> supervision from Dag, who helped steer the project in a great
>> direction to make it reusable (it's partially included in Numba and
>> will likely be in Theano in the future, hopefully others as well). I
>> also wrote a thesis for my master's, which can be found here
>> https://github.com/markflorisson88/minivect/tree/master/thesis, which
>> can shed
>> some light on some parts of the design and performance aspects.
>> Performance graphs can also be found here:
>> https://github.com/markflorisson88/minivect/tree/master/bench/graphs
>>
>> So anyway, how would you prefer dealing with the minivect submodule?
>> We could include it verbatim, with any modifications made to minivect
>> directly, since we'd have separate git histories. We could
>> alternatively make it an optional submodule which is only required
>> when actually using array expressions. I like the latter, but anything
>> is fine with me really.
>
> I think I support using a git submodule for now. Not sure about making
> it optional (which I assume would make array expression testcases not
> run if it minivect is not present so that there's no test failures); we
> want to make sure we are forced to do releases and testing right and
> include it, and if it is only required to compile code that uses
> memoryview expressions users could get confused about there being two
> Cython "editions" around.
>
> Since "git submodule" does link to a specific revision, so there's no
> stability concerns over verbatim inclusion.
>
> How hg-git deals with submodules is worth consideration too though.

Another option you didn't mention is to push the responsibility of 
getting minivect over to end-users; Cython simply tries to do "import 
minivect". This does have versioning issues though since git will likely 
not be the tool used to fetch the revision.

A lot more pain for those who uses array expressions, but a little less 
pain for the rest. So it depends on how you weigh the user groups.

Realistically, we'd want to depend on LLVM as well down the road for 
minivect stuff (at least if you want optimal performance), so perhaps 
opening the can-of-external-dependency-worms should just be done sooner 
rather than later.

Dag