zip() function troubles

[Raymond Hettinger]

On Jul 26, 4:25 pm, Istvan Albert <istvan.alb... at gmail.com> wrote:
> Now I know that zip () wastes lots of memory because it copies the
> content of the lists, I had used zip to try to trade memory for speed
> (heh!) , and now that everything was replaced with izip it works just
> fine.  What was really surprising is that it works with no issues up
> until 1 million items, but for say 10 million it pretty much goes
> nuts. Does anyone know why?

There's nothing in izip() that holds memory, tracks indices, or is
sensitive to the length of its inputs (it simply calls the next method
on the input iterators as returns the tuple result).  So, if you're
seeing behavior changes at 10 millions items, the cause almost
certainly lies elsewhere. One possible candidate could be memory
consumed by immortal integer objects.


Raymond Hettinger