Calling a generator multiple times

[Steven Majewski]

On Sat, 8 Dec 2001, Courageous wrote:

>
> >If it _was_ "plain wrong" it wouldn't be the way it is.
>
> One of the objections to what I've written has been, on several
> occasions, "After you've worked with them for a while, you'll
> not have any problems with them" or some equivalent. What's
> wrong-headed about this kind of thinking is that this is true
> of original programmers everywhere. They always generally
> understand their own code. Embedding a keyword deep in a function
> definition and then having this embedding make the function
> in effect behave entirely differently isn't just wrong, it's
> bizarre.
>

Normally, I'ld agree with you that 'once you get used to them'
isn't a good excuse. But, in the case of generators, I don't
think I've seen any proposals which make what's going on much
more clear. The problem is that there's a lot of syntactic magic
sugar dust being sprinkled here (something that Python usually
avoids) and so it's hard to capture the semantics accurately
with a simple syntax.


Semantically, this:

	def g(n):
		m = 0
		while n < m:
			yield n
			m += 1


is equivalent to something like this:

	class g(generator):

		def __init__(self,n):
		# make local vars into instance vars
		# and do the initial stuff before the loop
			self.n = n
			self.m = 0
		def next():
			tmp = self.m
			self.m += 1
			return tmp


 Function g() might be better named 'make_g()' . ( As Bruce Eckel noted,
it's a generator factory function ). In Lisp, we would implement something
like this as a macro 'make-generator, but Python doesn't handle that
type of code manipulation. ( The above is only an approximate translation
of the semantics of something that's implemented in a very different
manner. )

 In addition, there's really two levels of indirection going on to
confuse folks: there's the indirection in the generator definition,
which is really the definition of the generator that will be produced
by the syntactically invisible virtual generator factory, and there's
the indirection in the way the iteration protocol invisibly and
automagically calls the generator's 'next' method until StopIteration
is raised.

 And when you use these generators in a for loop, they two bits of
confusion seem to cancel each other out. "for i in g(10):" really
behaves just like it's definition.   I'm guessing that this fact
was one of the selling points for this syntax -- that if you didn't
look under the hood and just USED them, they would not be confusing.
I'm not sure that supposition has been born out, but they do get
less confusion with use.

 Perhaps it would be better to spend another keyword, along with
'def' and 'class' , to distinguish 'generator factories'. Besides
the fact that Guido and Python have both been conservative in
spending their keywords, I think one reason I haven't favored
any of the earlier suggestions was that they seemed to try to
stress a difference between generators and function, which seemed
to be an exaggerated difference. Generators, functions and methods
are just slight variations on each other. The real difference is
between functions and factories.

-- Steve Majewski