[Python-ideas] With clauses for generator expressions

[Andrew Barnert]

From: Steven D'Aprano <steve at pearwood.info>
Sent: Fri, November 16, 2012 1:53:42 AM


> > This seems to be an argument against with  statements, or any other kind of
> > resource management at all besides  "trust the GC".
> 
> Certainly not. I'm saying that for many applications,  explicit resource
> management is not critical -- letting the GC close the file  (or whatever
> resource you're working with) -- is a perfectly adequate  strategy. The mere
> existence of "faulty" gen expressions like the above  example is not
> necessarily a problem.
> 
> Think of it this way: you can  optimize code for speed, for memory, and for
> resource usage. (Memory of  course being a special case of resource usage.)
> You're worried about making  it easy to micro-optimize generator expressions
> for resource usage.

It's not a micro-optimization, or an optimization at all. It has nothing to do 
with performance, and everything to do with making your code work at all. (Or, 
in some cases, making it robust—your code may work 99% of the time, or work with 
CPython or POSIX but not PyPy or Windows.) For example, see Google's Python 
Style Guide 
at http://google-styleguide.googlecode.com/svn/trunk/pyguide.html#Files_and_Sockets
 for why they recommend always closing files.

> The only limitation here is that you can't use a context manager in a  list
> comprehension or generator expression.

Yes, that's exactly the limitation (but only in generator expressions—in list 
comprehensions, it can't ever matter).

> The beauty of generator expressions is that they  are deliberately lean. The
> bar to fatten them up with more syntax is quite  high, and I don't think you
> have come even close to getting over  it.

This is one of those cases where it won't hurt you when you don't use it. You 
don't have to put if clauses into generator expressions, or nest multiple 
loops—and very often you don't, in which case they don't get in the way, and 
your expression is concise and simple. Similarly, you won't have to put with 
clauses into generator expressions, and very often you won't, in which case they 
won't get in the way.

And I don't think anyone would have trouble learning or understanding it. The 
expression still maps to a generator function that's just a simple tree of 
one-line nested statements with a yield statement at the bottom, the only 
difference is that instead of the two most common kinds of statements in such 
functions, you can now use the three most common.