[Patches] [ python-Patches-1548388 ] set comprehensions

[SourceForge.net]

Patches item #1548388, was opened at 2006-08-29 04:33
Message generated for change (Comment added) made by gvanrossum
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1548388&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Core (C code)
Group: Python 3000
Status: Open
Resolution: None
Priority: 5
Submitted By: Georg Brandl (gbrandl)
Assigned to: Georg Brandl (gbrandl)
Summary: set comprehensions

Initial Comment:
This is a big one:

* cleanup grammar; unifies listcomp/genexp grammar
which means that [x for x in 1, 2] is no longer valid

* cleanup comprehension compiling code (unifies all AST
code for the three comprehensions and most of the
compile.c code)

* add set comprehensions

This patch modifies list comprehensions to be
implemented more like generator expressions: in a
separate function, which means that the loop variables
will not leak any more.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2006-09-06 11:36

Message:
Logged In: YES 
user_id=6380

I always assumed that the genexps *require* being a function
because that's the only way to create a generator.  But that
argument doesn't apply to listcomps.

That's about all I know of the implementation of these.. :-(

Have you asked python-dev?

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2006-09-06 03:03

Message:
Logged In: YES 
user_id=849994

It is complete, it works and it does not leak the loop
variable(s).

The question is whether it is okay for listcomps and
setcomps to be in their own anonymous function, which slows
listcomps down compared to the 2.x branch.

I don't know why the function approach was taken for
genexps, but I suspect it was because who implemented it
then saw this as the best way to hide the loop variable.

Perhaps somebody else more familiar with the internals and
the previous discussions can look over it.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2006-09-06 02:48

Message:
Logged In: YES 
user_id=6380

Do you think this is ready to be checked in, or are you
still working on it?

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2006-09-01 05:38

Message:
Logged In: YES 
user_id=849994

Since you can put anything usable as an assignment target
after the "for" of a listcomp, just renaming might be
complicated.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2006-08-31 19:40

Message:
Logged In: YES 
user_id=6380

+1.

Would this cause problems for abominations like this though?

>>> a=[1]
>>> list(tuple(a) for a[0] in "abc")
[('a',), ('b',), ('c',)]
>>> a
['c']
>>>


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2006-08-31 19:15

Message:
Logged In: YES 
user_id=80475

Would it be an oversimplfication for list and set comps to
keep everything in one code block and just hide the list
loop variables by renaming them:   x -->  __[x]

That approach would only require a minimal patch, and it
would make for a cleaner disassembly.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2006-08-31 15:55

Message:
Logged In: YES 
user_id=849994

Attaching slightly revised patch and bytecode comparison.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2006-08-29 18:30

Message:
Logged In: YES 
user_id=80475

Can you post a before and disassembly of some list and set
comprehensions.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2006-08-29 15:09

Message:
Logged In: YES 
user_id=849994

test_compiler and test_transformer fail because the compiler
package hasn't been updated yet.

test_dis fails because list comprehensions now generate
different bytecode.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2006-08-29 13:59

Message:
Logged In: YES 
user_id=6380

Nice!

I see failures in 4 tests:

    test_compiler test_dis test_transformer test_univnewlines

test_univnewlines is trivial (it's deleting a variable
leaked out of a list comprehension); haven't looked at the
rest in detail

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2006-08-29 04:34

Message:
Logged In: YES 
user_id=849994

The previously attached patch contains only the important
files. The FULL patch (attached now) also contains syntax
fixes in python files so that the test suite is mostly passing.

Note that the compiler package isn't ready yet.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1548388&group_id=5470