grokking generators and iterators

[Andrew Dalke]

Alex:
> It's a _bit_ more complicated because you have to construct an instance
> and stash away the state, but need be novere as complicated as in your
> example, I think.
   ..
>>   def __getitem__(self, i):
>>     # To allow use in for loops

> No need -- "being an iterator" in itself allows use in for
 > loops in Python 2.2.  __getitem__ was only needed for the
> purpose in 2.1 and earlier, where iterators didn't exist.

True.  I wrote what I did because I've needed to write something
which worked with 1.5.2 but I wanted it to also work with newer
Pythons appropriately.  Of course that also means my mind is stuck
in old solutions rather than rethinking things for modern solutions.

That's why I couldn't raise StopIteration

BTW, in my real code, I also had

  def __init__(self ...)
    ...
    self._n = 0
  def __getitem__(self i):
    assert self._n == i, "forward iteration only"

to prevent people from accidentally thinking this was a real list.


> >   def __iter__(self):
> >     # For new-style iters
> >     return iter(self, None)

> The two-arguments for of iter requires a callable as its
> first argument, and proceeds much like:
  ...
> So, this form of iter is not appropriate here.  Generally
 > a class whose instances are meant to BE iterators, as here,
> would just 'return self' in method __iter__.

Yep, that should have been
  return iter(self.next, None)

as used in StringIO.py and mailbox.py for this same purpose.
(Else .next() on the first "ends" with an infinite number of ""
empty strings, and in mailbox -- like mine -- ends with an infinite
number of None objects.)

(earlier)
> Method next should raise StopIteration when done,
> not just start returning an unending stream of None.

Method .next() of an iterator should raise StopIteration.  But
the container object itself doesn't need to implement the iteration
protocol.  Granted, it really should act the same, but I've written
many forward iteration objects which use the "end when you start
getting Nones back" protocol, so I'm glad the __iter__ solution and the
iter(callable, sentinel) function easily allow for that transition.

Quick scan of the standard library shows that multifile and pydoc
also do a "return a false value" to indicate that next() has nothing
more to yield.  I also recall scanning the stdlibs to see how others
did forward iteration through an object, so I'm not surprised that
they exist.

In any case, it *is* time for me to change my worn ways. :)

                    Andrew
                    dalke at dalkescientific.com