[Python-Dev] RE: Iterators, map, xreadlines and docs

[Tim Peters]

[Fred]
>   Won't there be at least one standard iterator object defined for
> lists, etc.?

>>> iter([])
<iterator object at 00792640>

>>> iter({})
<dictionary-iterator object at 00793A40>

>>> iter(())
<iterator object at 00792640>

>>> import sys
>>> iter(sys.stdin)
<callable-iterator object at 00793FE0>

>>> class C:
...     def __iter__(self): return self
...
>>> iter(C())
<__main__.C instance at 007938EC>
>>>

What do those have in common?  Objects and types are the wrong way to
approach this one:  it's really of no interest that, e.g., iter(list) and
iter(dict) return objects of different types; what *is* interesting is that
iter(whatever) returns *an* object that conforms to the iterator protocol (or
"implements the iterator interface" -- all the same to me).  You can give
*examples* of builtin iterator types that conform to the protocol, but the
protocol needs to be defined for its own sake first.  The protocol is
fundamental, and is neither an object nor a type.

> That could be described in the built-in types section (as with
> files, lists, etc.) of the Library Reference.  That will be used
> as the definition of the iterator protocol in the same way the
> file object description there is referred to from places that want
> file or file-like objects.

"file-like objects" is the bad doc experience I'm hoping we don't repeat.
The phrase "file-like object" is indeed used freely in the docs, but it's not
(AFAICT) *defined* anywhere, and doesn't even appear in the index.  Besides,
the notion that "file-like object" refers to section

    Buitin-in Types, Exceptions and Functions ->
     Other Built-in Types ->
      File Objects

was news to me.  I see the individual method descriptions there sometimes
refer to "file-like objects", and other times "objects implementing a
file-like interface".  The latter phrase appears uniquely in the description
of .readlines(), and may be the clearest explanation in the docs of wtf
"file-like object" means.  If so, it shouldn't be buried in the bowels of one
method description.

>   I think we need some re-organization of the built-in types section
> to separate abstract protocols from specific implementations,

Yes.

> but that's an orthagonal aspect and can be handled at the same
> time as the rest of the built-in types.

I assume you're thinking of creating a new "Iterator Objects" section under
"Other Built-in Types"?  That would work for me if it *started* with a
description of the iterator interface/protocol.

There's a twist, though:  iterators need to be defined already in the
Language Reference manual (we can't explain for-loops without them anymore).

>   Specific changes for places that accept iterators should be made as
> the code is changed, as usual.  Please describe the changes clearly in
> checkin messages so iterator related changes don't propogate to the
> maintenance branch.

We need an example to build on -- this is too abstract for me (which is
saying something <wink>).

For example, today we have:

    list(sequence)
        Return a list whose items are the same and in the same order as
        sequence's items. If sequence is already a list, a copy is made
        and returned, similar to sequence[:]. For instance, list('abc')
        returns returns ['a', 'b', 'c'] and list( (1, 2, 3) )
        returns [1, 2, 3].

list() doesn't yet work with iterators, but surely will.  What do we want the
docs to say after it changes?  Should it be implicit or explicit that
"sequence" now means "sequence or sequence-like object"?  Where is the
connection between "sequence-like object" and "iterable" explained?  Perhaps
what's really needed is s/sequence/iterable/ in this description.  But then
where is "iterable" defined?

Solve this once and the rest should follow easily.  But solving it the first
time doesn't look easy to me.  That's why I'm bugging you now.