Controlling a generator the pythonic way

[Terry Reedy]

"news:863bro4o9h.fsf at guru.mired.org...
> Thomas Lotze <thomas at thomas-lotze.de> writes:
>> A related problem is skipping whitespace. Sometimes you don't care about
>> whitespace tokens, sometimes you do. Using generators, you can either 
>> set
>> a state variable, say on the object the generator is an attribute of,
>> before each call that requires a deviation from the default, or you can
>> have a second generator for filtering the output of the first. Again, 
>> both
>> solutions are ugly (the second more so than the first).

Given an application that *only* wanted non-white tokens, or tokens meeting 
any other condition, filtering is, to me, exactly the right thing to do and 
not ugly at all.  See itertools or roll your own.

Given an application that intermittently wanted to skip over non-white 
tokens, I would use a *function*, not a second generator, that filtered the 
first when, and only when, that was wanted.  Given next_tok, the next 
method of a token generator, this is simply

def next_nonwhite():
   ret = next_tok()
   while not iswhte(ret):
      ret = next_tok()
   return ret

A generic method of sending data to a generator on the fly, without making 
it an attribute of a class, is to give the generator function a mutable 
parameter, a list, dict, or instance, which you mutate from outside as 
desired to change the operation of the generator.

The pair of statements
  <mutate generator mutable>
  val = gen.next()
can, of course, be wrapped in various possible gennext(args) functions at 
the cost of an additional function call.

Terry J. Reedy