[Numpy-discussion] nditer: possible to manually handle dimensions with different lengths?

[John Salvatier]

I apologize, I picked a poor example of what I want to do. Your suggestion
would work for the example I provided, but not for a more complex example.
My actual task is something like a "group by" operation along a particular
axis (with a known number of groups).

Let me try again: What I would like to be able to do is to specify some of
the iterator dimensions to be handled manually by me. For example lets say I
have some kind of a 2d smoothing algorithm. If I start with an array of
shape [a,b,c,d] and I'd like to do the 2d smoothing over the 2nd and 3rd
dimensions, I'd like to be able to tell nditer to do normal broadcasting and
iteration over the 1st and 4th dimensions but leave iteration over the 2nd
and 3rd dimensions to me and my algorithm. Each iteration of nditer would
give me a 2d array to which I apply my algorithm. This way I could write
more arbitrary functions that operate on arrays and support broadcasting.

Is clearer?

On Fri, Sep 30, 2011 at 5:04 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:

> On Fri, Sep 30, 2011 at 8:03 AM, John Salvatier <jsalvati at u.washington.edu
> > wrote:
>
>> Using nditer, is it possible to manually handle dimensions  with different
>> lengths?
>>
>> For example, lets say I had an array A[5, 100] and I wanted to sample
>> every 10 along the second axis so I would end up with an array B[5,10]. Is
>> it possible to do this with nditer, handling the iteration over the second
>> axis manually of course (probably in cython)?
>>
>> I want something like this (modified from
>> http://docs.scipy.org/doc/numpy/reference/arrays.nditer.html#putting-the-inner-loop-in-cython
>> )
>>
>> @cython.boundscheck(False)
>> def sum_squares_cy(arr):
>>     cdef np.ndarray[double] x
>>     cdef np.ndarray[double] y
>>     cdef int size
>>     cdef double value
>>     cdef int j
>>
>>     axeslist = list(arr.shape)
>>     axeslist[1] = -1
>>
>>     out = zeros((arr.shape[0], 10))
>>     it = np.nditer([arr, out], flags=['reduce_ok', 'external_loop',
>>                                       'buffered', 'delay_bufalloc'],
>>                 op_flags=[['readonly'], ['readwrite', 'no_broadcast']],
>>                 op_axes=[None, axeslist],
>>                 op_dtypes=['float64', 'float64'])
>>     it.operands[1][...] = 0
>>     it.reset()
>>     for xarr, yarr in it:
>>         x = xarr
>>         y = yarr
>>         size = x.shape[0]
>>         j = 0
>>         for i in range(size):
>>            #some magic here involving indexing into x[i] and y[j]
>>     return it.operands[1]
>>
>> Does this make sense? Is it possible to do?
>>
>
>  I'm not sure I understand precisely what you're asking. Maybe you could
> reshape A to have shape [5, 10, 10], so that one of those 10's can match up
> with the 10 in B, perhaps with the op_axes?
>
> -Mark
>
>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20111001/19296dbe/attachment.html>