efficiency of range() and xrange() in for loops

[Steve R. Hastings]

On Wed, 05 Apr 2006 22:24:24 +0200, Fredrik Lundh wrote:
>> If Python actually allocates the list, then clearly we should all use
>> "for i in xrange".
> 
> clearly we should all?  speak for yourself.

I apologize for my pithy turn of phrase.  Clearly what I should have
written was:

"If Python actually allocates the list, then clearly 'for i in xrange'
would be desirable in cases where memory allocation might possibly be a
problem; of course if you have lots of memory, or you are only looping a
small number of times, go ahead and use range() because why not."

I deeply apologize for inadvertently trying to tell you how to write a for
loop.  I hope the offense isn't unforgivable.


I apologise for the fault in the newsgroup posting.  Those responsible
have been sacked.


> the difference isn't very
> large, xrange is actually slower in some python versions, and you'll
> need the integer objects sooner or later anyway...

Actually, for many uses of "for i in (range|xrange)", you only need the
value of i, and you aren't doing anything with the integer object.  You
might not even be looking at the value of i:


start_time = time.time()
for i in xrange(10**6):
    run_some_function()
stop_time = time.time()
secs = (stop_time - start_time) / 10**6
print "run_some_function() took on average, %f seconds." % secs


In the above, clearly we should all use xrange()... oops, I meant, if you
want to, you could use xrange() instead of allocating a list of a million
integers and then doing nothing with the list itself and then deallocating
the list of a million integers.

I apologise again for the fault in the newsgroup posting.  Those
responsible for sacking the people who have just been sacked
have been sacked.




>> If Python doesn't currently optimize this case, is there any chance
>> this optimization could be added?
> 
> in Python 2.X, range is defined to return a list.  if you start
> returning something else, you'll break stuff.

Perhaps I'm mistaken here, but I don't see how this optimization could
possibly break anything.  range() makes a list, and for consumes it, and
the list isn't seen anywhere else.  If the Python compiler took this:

for i in range(10**6):
    pass


and produced code equivalent to this:

for i in iter(range(10**6))
   pass


How would this break stuff?
-- 
Steve R. Hastings    "Vita est"
steve at hastings.org    http://www.blarg.net/~steveha