[Python-ideas] With clauses for generator expressions

[Nick Coghlan]

On Fri, Nov 16, 2012 at 10:46 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> However, I realised there's a more serious problem with your idea: the
> outermost clause in a list comprehension or generator expression is
> evaluated immediately and passed as an argument to the inner scope that
> implements the loop, so you have an unresolved sequencing problem between
> the evaluation of that argument and the evaluation of the context manager.
> If you want the context manager inside the generator, you *can't* reference
> the name bound in the as clause in the outermost iterable.
>

(Andrew's reply here dropped the list from the cc, but I figure my
subsequent clarification is worth sharing more widely)

When you write a genexp like this:

    gen = (x for x in get_seq())

The expansion is *NOT* this:

    def _g():
        for x in get_seq():
            yield x

    gen = _g()

Instead, it is actually:

    def _g(iterable):
        for x in iterable:
            yield x

    gen = _g(get_seq())

That is, the outermost iterable is evaluated in the *current* scope, not
inside the generator. Thus, the entire proposal is rendered incoherent, as
there is no way for the context manager expression to be executed both
*before* the outermost iterable expression and *inside* the generator
function, since the generator doesn't get called until *after* the
outermost iterable expression has already been evaluated. (And, to stave of
the obvious question, no this order of evaluation is *not* negotiable, as
changing it would be a huge backwards compatibility breach, as well as
leading to a lot more obscure errors with generator expressions)

The reason PEP 403 is potentially relevant is because it lets you write a
one-shot generator function using the long form and still make it clear
that it *is* a one shot operation that creates the generator-iterator
directly, without exposing the generator function itself:

    @in gen = g()
    def g():
        for x in get_seq():
            yield x

Or, going back to the use case in the original post:

    @in upperlines = f()
    def f():
        with open('foo', 'r') as file:
            for line in file:
                yield line.upper()


Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121117/1ec34a7f/attachment.html>