[Python-ideas] With clauses for generator expressions

[Andrew Barnert]

First, I realize that people regularly propose with expressions. This is not the 
same thing.

The problem with the with statement is not that it can't be postfixed 
perl-style, or used in expressions. The problem is that it can't be used with 
generator expressions.

Here's the suggestion:

    upperlines = (lines.upper() for line in file with open('foo', 'r') as file)

This would be equivalent to:

    def foo():
        with open('foo', 'r') as file:
            for line in file:
                yield line.upper()
    upperlines = foo()

The motivation is that there is no way to write this properly using a with 
statement and a generator expression—in fact, the only way to get this right is 
with the generator function above. And almost nobody ever gets it right, even 
when you push them in the right direction (although occasionally they write a 
complex class that has the same effect).

That's why we still have tons of code like this lying around:

    upperlines = (lines.upper() for line in open('foo', 'r'))

Everyone knows that this only works with CPython, and isn't even quite right 
there, and yet people write it anyway, because there's no good alternative.

The with clause is inherently part of the generator expression, because the 
scope has to be dynamic. The file has to be closed when iteration finishes, not 
when creating the generator finishes (or when the generator is cleaned up—which 
is closer, but still wrong).

That's why a general-purpose "with expression" wouldn't actually help here; in 
fact, it would just make generator expressions with with clauses harder to 
parse. A with expression would have to be statically scoped to be general.

For more details, see this:

http://stupidpythonideas.blogspot.com/2012/11/with-clauses-for-generator-expressions.html