[Python-Dev] New PEP 342 suggestion: result() and allow "return with arguments" in generators (was Re: PEP 342 suggestion: start(), __call__() and unwind_call() methods)

[Nick Coghlan]

Nick Coghlan wrote:
> Although, if StopIteration.result was a read-only property with the above 
> definition, wouldn't that give us the benefit of "one obvious way" to return a 
> value from a coroutine without imposing any runtime cost on normal use of 
> StopIteration to finish an iterator?

Sometimes I miss the obvious. There's a *much*, *much* better place to store 
the return value of a generator than on the StopIteration exception that it 
raises when it finishes. Just save the return value in the *generator*.

And then provide a method on generators that is the functional equivalent of:

     def result():
         # Finish the generator if it isn't finished already
         for step in self:
             pass
         return self._result # Return the result saved when the block finished

It doesn't matter that a for loop swallows the StopIteration exception any 
more, because the return value is retrieved directly from the generator.

I also like that this interface could still be used even if the work of 
getting the result is actually farmed off to a separate thread or process 
behind the scenes.

Cheers,
Nick.

P.S. Here's what a basic trampoline scheduler without builtin asynchronous 
call support would look like if coroutines could return values directly. The 
bits that it cleans up are marked "NEW":

          import collections

          class Trampoline:
              """Manage communications between coroutines"""

              running = False

              def __init__(self):
                  self.queue = collections.deque()

              def add(self, coroutine):
                  """Request that a coroutine be executed"""
                  self.schedule(coroutine)

              def run(self):
                  result = None
                  self.running = True
                  try:
                      while self.running and self.queue:
                          func = self.queue.popleft()
                          result = func()
                      return result
                  finally:
                      self.running = False

              def stop(self):
                  self.running = False

              def schedule(self, coroutine, stack=(), call_result=None, *exc):
                  # Define the new pseudothread
                  def pseudothread():
                      try:
                          if exc:
                              callee = coroutine.throw(call_result, *exc)
                          else:
                              callee = coroutine.send(call_result)
                      except StopIteration: # NEW: no need to name exception
                          # Coroutine finished cleanly
                          if stack:
                              # Send the result to the caller
                              caller = stack[0]
                              prev_stack = stack[1]
                              # NEW: get result directly from callee
                              self.schedule(
                                   caller, prev_stack, callee.result()
                              )
                      except:
                          # Coroutine finished with an exception
                          if stack:
                              # send the error back to the caller
                              caller = stack[0]
                              prev_stack = stack[1]
                              self.schedule(
                                   caller, prev_stack, *sys.exc_info()
                              )
                          else:
                              # Nothing left in this pseudothread to
                              # handle it, let it propagate to the
                              # run loop
                              raise
                      else:
                          # Coroutine isn't finished yet
                          if callee is None:
                              # Reschedule the current coroutine
                              self.schedule(coroutine, stack)
                          elif isinstance(callee, types.GeneratorType):
                              # Make a call to another coroutine
                              self.schedule(callee, (coroutine,stack))
                          elif iscallable(callee):
                              # Make a blocking call in a separate thread
                              self.schedule(
                                   threaded(callee), (coroutine,stack)
                              )
                          else:
                              # Raise a TypeError in the current coroutine
                              self.schedule(coroutine, stack,
                                   TypeError, "Illegal argument to yield"
                              )

                  # Add the new pseudothread to the execution queue
                  self.queue.append(pseudothread)


P.P.S. Here's the simple coroutine that threads out a call to support 
asynchronous calls with the above scheduler:

   def threaded(func):
       class run_func(threading.Thread):
           def __init__(self):
               super(run_func, self).__init__()
               self.finished = False
           def run(self):
               print "Making call"
               self.result = func()
               self.finished = True
               print "Made call"
       call = run_func()
       call.start()
       print "Started call"
       while not call.finished:
           yield # Not finished yet so reschedule
       print "Finished call"
       return call.result

I tried this out by replacing 'yield' with 'yield None' and 'return 
call.result' with 'print call.result':

Py> x = threaded(lambda: "Hi there!")
Py> x.next()
Started call
Making call
Made call
Py> x.next()
Finished call
Hi there!
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
StopIteration

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com