Iterators, iterables and special objects

[Terry Reedy]

On 7/23/2020 5:14 AM, Peter Slížik wrote:
>> Works in what way? You can't use it in a 'for' loop if it doesn't
>> define __iter__.
>>
> 
> class Iterable:
>      def __iter__(self):
>          return Iterator(...)
> 
> class Iterator:
>      def __next__(self):
>          return <next item>
> 
>      # No __iter__ here.
>      # I've just forgotten to def it.

Which means that an Iterator is not an iterator.

> With this setup, using for to iterate over Iterable *will* still work,
> though you cannot use the trick described below.
> 
>> Yes? It is indeed an iterator, since its iter method returns itself. It
> is also iterable, since it has an iter method.
> Of course, you're right. But you're reading the article through the eyes of
> an experienced developer.
> 
> 
>> It's so you can do things like this:
>>
>> def show_with_header(stuff):
>>      stuff = iter(stuff)
>>      hdr = next(stuff)
>>      print(hdr)
>>      for line in stuff:
>>          print(line)
>>          if new_page(): print(hdr)
>>
> 
> Yes, @Terry had given the same example. Frankly, I didn't know about it
> before, I somehow had the misconception that for always got a 'fresh'
> iterator...

The way 'for' gets its iterator is to call iter(iterable).  If the 
iterable is an iterator, iter(iterator) returns iterator.  Open files 
are *iterators*.

 >>> file.__iter__() is file
True

This means that one can do things like

file  = open(...)
for line in file:  # Section 1 of file.
     if separator(line): break
     process1(line)
process_sep(line)  # Special line that broke iteration.
for line in file:  # Section 2 of file.
     process2(line)

One alternative is something like

file = open(...)
section = 1
for line in file:
     if section == 1:
	if separator(line):
             process_sep(line)
             section = 2
             continue
         process1(line)
         continue
     else:
         process2(line)

Another is to use while loops and explicit next(line) calls.  The ':=' 
assignment operator makes this a bit easier.  I used to commonly process 
files with multiple sections and different 'key' lines marking them.

-- 
Terry Jan Reedy