[Python-Dev] PEP 380 - return value question and prototype implementation (was Thoughts fresh after EuroPython)
P.J. Eby
pje at telecommunity.com
Sun Jul 25 03:51:37 CEST 2010
At 07:08 AM 7/24/2010 -0700, Guido van Rossum wrote:
>- After seeing Raymond's talk about monocle (search for it on PyPI) I
>am getting excited again about PEP 380 (yield from, return values from
>generators). Having read the PEP on the plane back home I didn't see
>anything wrong with it, so it could just be accepted in its current
>form.
I would like to reiterate (no pun intended) the suggestion of a
special syntactic form for the return, such as "yield return x", or
"return with x" or something similar, to distinguish it from a normal
generator return.
I think that when people are getting used to the idea of generators,
it's important for them to get the idea that the function's "return
value" isn't really a value, it's an iterator object. Allowing a
return value, but then having that value silently disappear, seems
like it would delay that learning, so, a special form might help to
make it clear that the generator in question is intended for use with
a corresponding "yield from", and help avoid confusion on this.
(I could of course be wrong, and would defer to anyone who sees a
better way to explain/teach around this issue. In any event, I'm +1
on the PEP otherwise.)
By the way, the PEP's "optimized" implementation could probably be
done just by making generator functions containing yield-from
statements return an object of a different type than the standard
geniter. Here's a Python implementation sketch, using a helper class
and a decorator -- translation to a C version is likely
straightforward, as it'll basically be this plus a light sprinkling
of syntactic sugar.
So, in the pure-Python prototype (without syntax sugaring), usage
would look like this:
@From.container
def some_generator(...):
...
yield From(other_generator(...)) # equivalent to 'yield from'
...
def other_generator(...):
...
raise StopIteration(value) # equivalent to 'return value'
We mark some_generator() with @From.container to indicate that it
uses 'yield from' internally (which would happen automatically in the
C/syntax sugar version). We don't mark other_generator(), though,
because it doesn't contain a 'yield from'.
Now, the implementation code (a slightly altered/watered-down version
of a trampoline I've used before in 2.x, hopefully altered correctly
for Python 3.x syntax/semantics):
class From:
@classmethod
def container(cls, func):
def decorated(*args, **kw):
return cls(func(*args, **kw)) # wrap generator in a
From() instance
return decorated
def __new__(cls, geniter):
if isinstance(geniter, cls):
# It's already a 'From' instance, just return it
return geniter
self = object.__new__(cls, geniter)
self.stack = [geniter]
return self
def __iter__(self):
return self
def __next__(self):
return self._step()
def send(self, value):
return self._step(value)
def throw(self, *exc_info):
return self._step(None, exc_info)
def _step(self, value=None, exc_info=()):
if not self.stack:
raise RuntimeError("Can't resume completed generator")
try:
while self.stack:
try:
it = self.stack[-1]
if exc_info:
try:
rv = it.throw(*exc_info)
finally:
exc_info = ()
elif value is not None:
rv = it.send(value)
else:
rv = it.next()
except:
value = None
exc_info = sys.exc_info()
if exc_info[0] is StopIteration:
# pass return value up the stack
value, = exc_info[1].args or (None,)
exc_info = () # but not the error
self.stack.pop()
else:
if isinstance(rv, From):
stack.extend(rv.stack) # Call subgenerator
value, exc_info, rv = None, (), None
else:
return rv # it's a value to yield/return
else:
# Stack's empty, so exit w/current return value or error
if exc_info:
raise exc_info[1]
else:
return value
finally:
exc_info = () # don't let this create garbage
def close(self):
if self.stack:
try:
# There's probably a cleaner way to do this in Py 3, I just
# don't know what it is off the top of my head...
raise GeneratorExit
except GeneratorExit as e:
try:
self.throw(*sys.exc_info())
except (StopIteration, GeneratorExit):
pass
else:
raise RuntimeError("Generator(s) failed to close()")
# XXX probably needs some more code here to clean up stack
def __del__(self):
try:
self.close()
except:
pass
As you can (hopefully) see, the main code path simply ends up
delegating next/send/close etc. directly to the iterator on the top
of the stack, so there isn't any multi-layer passthru going on, as in
the "non-optimized" version of the spec in the PEP.
More information about the Python-Dev
mailing list