[Numpy-discussion] Fancy Indexing of Structured Arrays is Slow

[Dave Hirschfeld]

Sebastian Berg <sebastian <at> sipsolutions.net> writes:

> 
> On Do, 2014-05-15 at 12:31 +0000, Dave Hirschfeld wrote:
> > As can be seen from the code below (or in the notebook linked beneath) 
fancy 
> > indexing of a structured array is twice as slow as indexing both fields 
> > independently - making it 4x slower?
> > 
> > 
> 
> The non-vanilla types tend to be somewhat more efficient with these
> things and the first indexing does not copy so it is rather fast. I did
> not check the code, but we use (also in the new one for this operation)
> the copyswap function on individual elements (only for non-trivial
> copies in 1.9 in later, making the difference even larger), and this is
> probably not specialized to the specific void type so it probably has to
> do call the copyswap for every field (and first get the fields). All
> that work would be done for every element.
> If you are interested in this, you could check the fancy indexing inner
> loop and see if replacing the copyswap with the specialized strided
> transfer functions (it is used further down in a different branch of the
> loop) actually makes things faster. I would expect so for some void
> types anyway, but not sure in general.
> 
> - Sebastian
> 

Thanks for the explanation and pointers - it sounds like a good opportunity 
for getting stuck into the internals of numpy which I've been meaning to do. 
I'm not sure I've got the required skills but I'm sure it will be a good 
learning experience.

Unfortunately it won't likely be in the immediate future that'll I'll have 
the time to do so. In the meantime I can live with indexing the fields 
independently.

Thanks,
Dave