[Numpy-discussion] MapIter api

[Sebastian Berg]

On Tue, 2013-04-23 at 17:08 -0400, Frédéric Bastien wrote:
> Hi,
> 
> this is currently used in Theano! In fact, it is a John S. that
> implemented it in NumPy to allow fast gradient of the advanced
> indexing in Theano. It allow code like:
> 
> 
> matrix1[vector1, vector2] += matrix2
> 
Yes, I had missed that and thought maybe nobody actually used it yet. I
gave some points why I think there should be some changes in the
original pull request [1]. Mostly I think it would make sense (also a
lot for theano) to rewrite it with the new iterators and expose the
subspace more directly. That would give vast speedups for mixed
fancy/non-fancy indices.

But if this is useful to you, I guess one can also just create a new one
if someone finds time, leaving the old MapIter deprecated and
unmaintained.

[1] https://github.com/numpy/numpy/pull/377

> where there is duplicate indices in the vector
> 
> In looking at the code, I saw it use at least those part of the
> interface.
> 
> PyArrayMapIterObject
> PyArray_MapIterNext
> PyArray_ITER_NEXT
> PyArray_MapIterSwapAxes
> PyArray_BroadcastToShape
> 

There is likely no reason for changing these, but improving MapIter
would likely break binary compatibility because of struct access.

- Sebastian
> 
> I lost the end of this discussion, but I think this is not possible in
> NumPy as there was not an agreement to include that. But I remember a
> few other user on this list asking for this(and they where Theano user
> to my knowledge).
> 
> 
> So I would prefer that you don't remove the part that we use for the
> next 1.8 release.
> 
> thanks
> 
> Frédéric
> 
> 
> 
> On Tue, Apr 16, 2013 at 9:54 AM, Nathaniel Smith <njs at pobox.com>
> wrote:
>         On Mon, Apr 15, 2013 at 5:29 PM, Sebastian Berg
>         <sebastian at sipsolutions.net> wrote:
>         > Hey,
>         >
>         > the MapIter API has only been made public in master right?
>         So it is no
>         > problem at all to change at least the mapiter struct, right?
>         >
>         > I got annoyed at all those special cases that make things
>         difficult to
>         > get an idea where to put i.e. to fix the boolean array-like
>         stuff. So
>         > actually started rewriting it (and I already got one big
>         function that
>         > does all index preparation -- ok it is untested but its
>         basically
>         > there).
>         >
>         > I would guess it is not really a big problem even if it was
>         public for
>         > longer, since you shouldn't do those direct struct access
>         probably? But
>         > just checking.
>         
>         
>         Why don't we just make the struct opaque, i.e., just declare
>         it in the
>         public header file and move the actual definition to an
>         internal
>         header file?
>         
>         If it's too annoying I guess we could even make it non-public,
>         at
>         least in 1.8 -- IIRC it's only there so we can use it in
>         umath, and
>         IIRC the patch to use it hasn't landed yet. Or we could just
>         merge
>         umath and multiarray into a single .so, that would save a
>         *lot* of
>         annoying fiddling with the public API that doesn't actually
>         serve any
>         purpose.
>         
>         -n
>         _______________________________________________
>         NumPy-Discussion mailing list
>         NumPy-Discussion at scipy.org
>         http://mail.scipy.org/mailman/listinfo/numpy-discussion
>         
> 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion