[SciPy-User] Populating a recarray from 0 size

[Christopher Barker]

Sergi Pons Freixes wrote:
> The problem is that the size is not known. I'll think about the list of tuples.

It is the generally accepted method for building up an array -- build a 
list first, then convert to an array. However, I think it would be great 
to have a built-in way to do this with numpy efficiently, particularly 
for things like recarrays and data types that pure python does not have.

> I would also appreciate is a reasoning of why is scipy.resize (e.g.,
> numpy.resize) raising this exception.

true -- probably a bug/missing feature. However, calling resize on every 
addition is likely to be slow and inefficient anyway -- it has to 
re-allocate memory every single time.

The way this is usually handled (and is handled by python lists...) is 
to pre-allocate more than required at first, and then when that amount 
gets full, re-allocate some more -- this way you only re-allocate every 
once in a while, rather than at every append -- at the expense of some 
wasted memory.

With that in mind, I wrote a pure python numpy array "accumulator", 
which used .resize() under the hood, but pre-allocates. The performance 
isn't' great, but should be better than calling a re-size() every time. 
I'd like to see something like this written in C some day -- but who 
knows if I'll ever get around to it...

Enclosed, if you want to try it out.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Accumulator.zip
Type: application/zip
Size: 4701 bytes
Desc: not available
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20100822/cbb8e52e/attachment.zip>