[Python-Dev] Python3 compiled listcomp can't see local var - bug or feature?

[Rob Cliffe]

Is this a bug or a feature?
Consider the following program:

# TestProgram.py
def Test():
   # global x
     x = 1
     exec(compile('print([x+1,x+2])', 'MyTest', 'exec'))
     exec(compile('print([x+i for i in range(1,3)])', 'MyTest', 'exec'))
Test()

In Python 2.7.15 the output is

[2, 3]
[2, 3]

In Python 3.6.5 the output is
[2, 3]
Traceback (most recent call last):
   File "TestProgram.py", line 7, in <module>
     Test()
   File "TestProgram.py", line 6, in Test
     exec(compile('print([x+i for i in range(1,3)])', 'MyTest', 'exec'))
   File "MyTest", line 1, in <module>
   File "MyTest", line 1, in <listcomp>
NameError: name 'x' is not defined

If the "global x" declaration is uncommented, this "fixes" the Python 
3.6.5 behaviour,
i.e. no error occurs and the output is the same as for Python 2.7.15.

*In other words, it looks as if in Python 3.6.5, the compiled list 
comprehension**
**can "see" a pre-existing global variable but not a local one.*

I have used dis to examine the code objects returned by compile()
(they are the same with or without the "global x"):

Python 2.7.15 first code object from 'print([x+1,x+2])':
   1           0 LOAD_NAME                0 (x)
               3 LOAD_CONST               0 (1)
               6 BINARY_ADD
               7 LOAD_NAME                0 (x)
              10 LOAD_CONST               1 (2)
              13 BINARY_ADD
              14 BUILD_LIST               2
              17 PRINT_ITEM
              18 PRINT_NEWLINE
              19 LOAD_CONST               2 (None)
              22 RETURN_VALUE
Python 2.7.15 second code object from 'print([x+i for i in range(1,3)])':
   1           0 BUILD_LIST               0
               3 LOAD_NAME                0 (range)
               6 LOAD_CONST               0 (1)
               9 LOAD_CONST               1 (3)
              12 CALL_FUNCTION            2
              15 GET_ITER
         >>   16 FOR_ITER                16 (to 35)
              19 STORE_NAME               1 (i)
              22 LOAD_NAME                2 (x)
              25 LOAD_NAME                1 (i)
              28 BINARY_ADD
              29 LIST_APPEND              2
              32 JUMP_ABSOLUTE           16
         >>   35 PRINT_ITEM
              36 PRINT_NEWLINE
              37 LOAD_CONST               2 (None)
              40 RETURN_VALUE
Python 3.6.5 first code object from 'print([x+1,x+2])':
   1           0 LOAD_NAME                0 (print)
               2 LOAD_NAME                1 (x)
               4 LOAD_CONST               0 (1)
               6 BINARY_ADD
               8 LOAD_NAME                1 (x)
              10 LOAD_CONST               1 (2)
              12 BINARY_ADD
              14 BUILD_LIST               2
              16 CALL_FUNCTION            1
              18 POP_TOP
              20 LOAD_CONST               2 (None)
              22 RETURN_VALUE
Python 3.6.5 second code object from 'print([x+i for i in range(1,3)])':
   1           0 LOAD_NAME                0 (print)
               2 LOAD_CONST               0 (<code object <listcomp> at 
0x00000000029F79C0, file "MyTest", line 1>)
               4 LOAD_CONST               1 ('<listcomp>')
               6 MAKE_FUNCTION            0
               8 LOAD_NAME                1 (range)
              10 LOAD_CONST               2 (1)
              12 LOAD_CONST               3 (3)
              14 CALL_FUNCTION            2
              16 GET_ITER
              18 CALL_FUNCTION            1
              20 CALL_FUNCTION            1
              22 POP_TOP
              24 LOAD_CONST               4 (None)
              26 RETURN_VALUE

You will see that in Python 3.6.5 the dis output for the second code object
does not show the internals of the listcomp, and in particular whether,
and how, it refers to the variable 'x'.  I don't know how to investigate 
further.

Best wishes
Rob Cliffe

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180606/a75c45a5/attachment.html>