Two candies

[Raymond Hettinger]

[Raymond]
> >#3 is a fine choice.  It is memory efficient -- the repeat() itertool takes-up only a few bytes.  It doesn't need psyco, you already have to fast C routines talking to each other without having to go through the interpreter loop.<

[bearophile]
> In my code I have found otherwise, so to be more sure I have just done
> few benchmarks on a Pentium3 CPU, 256 MB RAM, ActivePython 2.5.1.1, on
> Win.

I applaud your efforts to measure performance -- that is a reasonably
good way to find-out the best approach (though you have to be very
careful about what you measure, how you measure it, that the system
state hasn't changed between measurements, and how your interpret the
results).

Here's a few thoughts based on knowing what's under-the-hood.

* xrange() and repeat() objects only take-up a few bytes.

* xrange() creates a new integer object for every iteration

* repeat() will re-use the same object over and over

* the counter for repeat() does not use integer objects

* the list approach takes 4 bytes of memory per entry (pointers to a
single object)

* lists, xrange objects, and repeat objects all support the iteration
protocol

* lists and xrange objects also support the sequence protocol (getitem
and len)

* array_new has a special case for lists -- it uses the known length
to pre-size the array and it uses the getitem protocol to access the
elements

* array_new handles repeat() and xrange() by using the iterator
protocol and it grows the array one element at a time, with periodic
resizing and over-allocation -- this is dog slow

* in most apps (except for sparse arrays), the initialization time for
an array is dominated by the time spent actually doing something
useful with the array (iow, this is an odd place to be optimizing)

* the array module could be made a little smarter by checking the
iterators for a hint about their size (this would speed-up the
xrange() and repeat() versions considerably).

* the array module is not designed for speed -- it is all about
storing data compactly

* in contract, numpy and other numerical apps are more thoroughly
optimized

* memory consumption is very difficult to measure since the operating
system has a say in the matter (ask for 1 byte of allocation and you
may get an enormous chunk in return).

That being said, I'll cut to the chase:

* sometimes its easy to get so wrapped-up in thinking this though,
that the obvious gets missed.

* try adding this one to your test suite:

    a = array('l', [0]) * n

Raymond