[Python-ideas] With clauses for generator expressions

[Andrew Barnert]

Ah, you're right. The only way this would work is if the with clause were second or later, which would be very uncommon. And the fact that it doesn't work in the most common case means that, even if it were occasionally useful, it would cause a lot more confusion than benefit.

So, never mind...

Sent from my iPhone

On Nov 16, 2012, at 7:53, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Fri, Nov 16, 2012 at 10:46 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> However, I realised there's a more serious problem with your idea: the outermost clause in a list comprehension or generator expression is evaluated immediately and passed as an argument to the inner scope that implements the loop, so you have an unresolved sequencing problem between the evaluation of that argument and the evaluation of the context manager. If you want the context manager inside the generator, you *can't* reference the name bound in the as clause in the outermost iterable.
> 
> (Andrew's reply here dropped the list from the cc, but I figure my subsequent clarification is worth sharing more widely) 
> 
> When you write a genexp like this:
> 
>     gen = (x for x in get_seq())
> 
> The expansion is *NOT* this:
> 
>     def _g():
>         for x in get_seq():
>             yield x
> 
>     gen = _g()
> 
> Instead, it is actually:
> 
>     def _g(iterable):
>         for x in iterable:
>             yield x
> 
>     gen = _g(get_seq())
> 
> That is, the outermost iterable is evaluated in the *current* scope, not inside the generator. Thus, the entire proposal is rendered incoherent, as there is no way for the context manager expression to be executed both *before* the outermost iterable expression and *inside* the generator function, since the generator doesn't get called until *after* the outermost iterable expression has already been evaluated. (And, to stave of the obvious question, no this order of evaluation is *not* negotiable, as changing it would be a huge backwards compatibility breach, as well as leading to a lot more obscure errors with generator expressions)
> 
> The reason PEP 403 is potentially relevant is because it lets you write a one-shot generator function using the long form and still make it clear that it *is* a one shot operation that creates the generator-iterator directly, without exposing the generator function itself:
> 
>     @in gen = g()
>     def g():
>         for x in get_seq():
>             yield x
> 
> Or, going back to the use case in the original post:
> 
>     @in upperlines = f()
>     def f():
>         with open('foo', 'r') as file:
>             for line in file:
>                 yield line.upper()
> 
> 
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121116/bc684235/attachment.html>