Python is faster than C

[Armin Rigo]

Hello Paul,

Paul Rubin wrote:
> I think enumerate(xrange(1000000000)) returning a normal list would
> exhaust memory before some later optimizer had a chance to do anything
> with it.

There are two levels: the language specification and its implementation.
My point is that there is no reason why everything that is (at the
language level) a list, should always be implemented as a plain array of
objects.  The list returned by range(1000000) doesn't have to use 4MB of
memory when the same information can be encoded in a couple of ints. 
The presence of xrange() is the oldest of these user-visible
optimization hack that should have been optimized in the implementation
instead.  Similarily, if you do 'a'*999999999 I can think of a better
way to encode the resulting string than with 100MB of RAM.

On your specific example, we could also argue that a slightly involved
but reasonable amount of effort would allow C functions to internally
receive a context hint, which would tell enumerate() that its result
will only ever be used for iteration when this is the case, so that it
can internally return an iterable instead of a list -- but this would
only be a hack, a workaround for the limits of CPython's internal object
representations.


Armin