[Numpy-discussion] Histograms via indirect index arrays

[Travis Oliphant]

Piotr Luszczek wrote:

>>Yes, this is intended (sort of --- this particular example isn't the
>>reason for the behavior though).
>>
>>The issue is that the code g[idx] +=1 is equivalent in Python to
>>
>>g[idx] = g[idx] + 1
>>    
>>
>
>This is not what I read at:
>
>http://docs.python.org/ref/numeric-types.html
>
>Quote:
>
>These methods should attempt to do the operation in-place (modifying 
>self) and return the result (which could be, but does not have to be, 
>self).
>
>What you describe is "lack of attempt" in which case the "fall back"
>behavior gets triggered.
>  
>

The problems is that this explanation is very clear when we are talking 
about the syntax

g += 1

Then, there is a method of g that can be over-ridden to get the desired 
behavior.  Now, what you are talking about is "indexing followed by 
in-place addition".

i.e.  at issue is

how does python interpret

g[idx] += 1

How does this get compiled to byte-code?

There are two possibilities:

1) g[idx] creates a new object which then has 1 added to it using 
in-place addition.

     This would not produce the desired behavior as g[idx] is a copy of 
the data when idx is a
      general indexing array as it is in this case.  So, you make a copy 
of those indices, add 1 to them
      and then do what with the resut?

2) g[idx] += 1  gets converted to g[idx] = g[idx] + 1

    This appears to be effectively what Python actually does.  Notice 
that there is no way for us to control this behavior because there is no 
__inplace_with_indexing_add__ operator to over-ride.

There is no such single operation to over-ride for the object.   In 
other words, I don't see anyay for us to even alter the object to get 
the behavior you want from that syntax.  We can, of course, add a 
function or method to do that, but I we would have to extend Python to 
get the behavior you want here.


Note, however, if idx is slice syntax, then the operation is done 
without making copies because, for example,

g[1:10:2]

returns a "view" of the data (an array that shares the same memory) as 
the original array.  Thus,

g[1:10:2] += 1

does an inplace add without data-copying.

It may be possible to do what you want using zero-strided arrays, but 
I'll leave that for a more motivated contributor.

-Travis