iterating over a file with two pointers

[Peter Otten]

Oscar Benjamin wrote:

> $ cat tee.py
> #!/usr/bin/env python
> 
> import sys
> from itertools import tee
> 
> items = iter(range(int(sys.argv[1])))
> 
> while True:
>     for x in items:
>         items, discard = tee(items)
>         break
>     else:
>         break
> 
> print(x)
> 
> $ time py -3.3 ./tee.py 100000000
> 99999999
> 
> real    1m47.711s
> user    0m0.015s
> sys     0m0.000s
> 
> While running the above python.exe was using 6MB of memory (according
> to Task Manager). I believe this is because tee() works as follows
> (which I made up but it's how I imagine it).

[...]

> However, when I ran the above script on Python 2.7 it did consume
> massive amounts of memory (1.6GB) and ran slower so maybe this depends
> on optimisations that were introduced in 3.x.

Did you use xrange()?