[Python-ideas] Consider making enumerate a sequence if its argument is a sequence

[Steven D'Aprano]

On Thu, Oct 01, 2015 at 08:15:25AM +0300, Akira Li wrote:

> True or false?: do all iterables return the same items twice?
>   http://www.fallacyfiles.org/loadques.html

[Aside: I have no idea what point you are making with the above link.]

Of course they don't necessarily do so, but those that don't are not 
necessarily well-behaved. 

In the case of sequences and collections, the concept is that (absent 
any explicit mutation operation), iterating over it twice *should* give 
the same results, that is the normal expectation. But that isn't 
enforced, we can write something that breaks that rule:

class WeirdIterable:
    def __getitem__(self, i):
        if random.random() > 0.9: raise IndexError
        return random.choice(["fe", "fi", "fo", "fum"])

but most people would consider that to be a pathological case. Yes, you 
can do it, and maybe you have a reason to do so, but you can't expect 
other people's code to deal with it gracefully.

In the case of iterators, the answer is *certainly not*.

Iterators are designed for the express purpose of handling not just the 
"lazy sequence" case where you choose to calculate results on demand as 
an optimization, but the case where you *have no choice* because the 
results are coming from some source which may change from run to run, 
e.g. an external data source. An iterator *may* repeat if run twice, but 
there is no expectation that it will do so. It's not just that the rule 
about repeatability is not enforced, but that there is no such rule in 
the first place.

(By the way, when I talk about running an iterator twice, I'm completely 
aware that technically you cannot ever do so. What I mean is to iterate 
over the object, then *recreate the object* in some sense, then iterate 
over it again.)


> Specific application may use more specific requirements e.g.:
> 
> list(iterable):
> 
> - does it mean that all iterables must be finite?
> - do we need a special word to describe what list() accepts?

No, and no.

In principle, list() will quite happily create an infinite list for you, 
if you have infinite memory :-) The fact that in practice lists are 
probably limited to something of the order of 2**64 items or less is a 
mere quality of implementation issue :-)

But to be more serious, no, in context we should understand that lists 
have actual physical limits, and even finite iterables may not be 
capable of being turned into lists:

def gen():
    for i in range(10**10000):
        yield i

Perfectly finite in size, but you cannot have a list that big. It's not 
just *infinite iterables* which are prohibited, that's just a special 
case of iterables that will provide more items than you have memory to 
store. And that's not a fixed limit, it will differ from machine to 
machine.


[...]
> You've got the idea: the word *iterable* may be used in the context when
> not all iterables are accepted.

Sure. But the distinction is that while there are a whole lot of 
different iterables:

- iterables with a sufficiently small number of items
- iterables of hashable items
- iterables of (hashable key, item) pairs
- iterables of prime numbers less than one million
- iterables of strings containing exactly 1 vowel

etc they are special cases and don't need specialised names. But there 
is a *general* distinction between two cases:

- iterables which are iterators
- iterables which are not iterators

We have a name for the first set: "iterators". But we don't have a name 
for the second set. Andrew suggested "non-iterator iterables" is too 
clumsy for general use, and suggests we need a better name. You 
suggested "iterables", but that clearly cannot work, since iterators are 
a kind of iterable.




-- 
Steve