[Python-3000] Efficient LC's which don't leak the iteration variables (was Re: Py3k release schedule worries)

[Guido van Rossum]

On 12/20/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Georg Brandl wrote:
> > I'll implement it if only someone can point me in the right direction
> > how to do it.
>
> One idea I've had on that front is to persuade the compiler to replace the
> names used for the iteration variables in the source code either with the
> compiler's own hidden variable names (like the ones it already uses for
> various internal results it can't leave on the stack), or else to add the idea
> of a 'scope prefix' that assigns names in the existing function scope with a
> numeric prefix in front of each of the symbols.
>
> I haven't actually tried to implement that, though, and I suspect things will
> get a little tricky when it comes to dealing with nested scopes like '[(lambda
> i=i: i) for i in range(10)]'.

That doesn't strike me as tricky; I expect it'll happen automatically
if you do the right thing.

Trickier seems to be the possibility of writing

a = {}
for a[0] in ...: ...

We should probably just leave this and other similar anomalies alone.

> Another alternative might be to flag each list comprehension iteration
> variable during the symtable pass as being either deleted or restored when the
> comprehension is complete. Then before the LC emit code to save any variables
> to be restored to a hidden compiler variable and code after the LC to restore
> them to their original values. For deleted variables, emit the relevant
> deletion opcode after the LC.

Delete/restore seems utterly wrong with me -- it wouldn't work if one
of the variables affected is a cell referenced by a nested function
that was previously defined and that happens to be called at any time
during the iteration.

I think that the systematic renaming should work though, and be
relatively easy -- you just have to modify all of the expression
compiler to be aware of the potential renaming.

> This second approach, however, still has issues dealing with nested scopes (in
> particular closure variables will refer to the variable at function scope
> instead of the final value of the iteration variable). In addition, it needs
> to be able to handle conditionally defined variables: ones which may or may
> not be defined when the LC executes depending on which path was followed
> through earlier parts of the function.
>
> So I think the first approach I mentioned (particularly the 'scope prefix'
> concept) is the one most likely to prove workable. But YMMV, since I haven't
> actually *tried* any of this.

Neither have I. But experiments should be easy enough.

(Thanks Nick for the helpful suggestions!)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)