Memory leak while looping

[Alex Martelli]

Brandon Beck wrote:

> It appears your "leak" is with the call to range().  range(1, 10000000)
> causes a list of 10000000 to be created in memory.  This has nothing to
> do with item being assigned in the body of the loop, and you should see
> the same behavior in a script without the for loop and just the call to
> range.  Consider using the generator (or maybe it's just generator like)
> version of range, xrange.  That should run in constant memory.

Yes.  xrange is "just generator-like".  If you only need to repeat the
action N times and don't need the loop index in the body, consider also
Python 2.3's itertools.repeat (if you can afford to wait for Python 2.3).

Here are some measurements with Python 2.3 alpha on my old Athlon 1.2GHz
machine with Linux...:

[alex at lancelot python2.3]$ python timeit.py -s'import itertools as it' 'for 
x in range(1000*1000): pass'
10 loops, best of 3: 2.99e+05 usec per loop
[alex at lancelot python2.3]$ python timeit.py -s'import itertools as it' 'for 
x in xrange(1000*1000): pass'
10 loops, best of 3: 1.06e+05 usec per loop
[alex at lancelot python2.3]$ python timeit.py -s'import itertools as it' 'for 
x in it.repeat(0,1000*1000): pass'
10 loops, best of 3: 8.68e+04 usec per loop
[alex at lancelot python2.3]$

i.e., range's overhead for a million items is 300 milliseconds, xrange's
is 100 milliseconds, itertools.repeat's is 87 milliseconds -- and if you
are in a truly serious bottleneck, every little bit helps;-).


Alex