exec() an locals() puzzle

[Martin Di Paola]

I did a few tests

# test 1
def f():
     i = 1
     print(locals())
     exec('y = i; print(y); print(locals())')
     print(locals())
     a = eval('y')
     print(locals())
     u = a
     print(u)
f()

{'i': 1}
1
{'i': 1, 'y': 1}
{'i': 1, 'y': 1}
{'i': 1, 'y': 1, 'a': 1}
1

# test 2
def f():
     i = 1
     print(locals())
     exec('y = i; print(y); print(locals())')
     print(locals())
     a = eval('y')
     print(locals())
     y = a
     print(y)
f()
{'i': 1}
1
{'i': 1, 'y': 1}
{'i': 1}
Traceback (most recent call last):
NameError: name 'y' is not defined


So test 1 and 2 are the same except that the variable 'y' is not
present/present in the f's code.

When it is not present, exec() modifies the f's locals and adds an 'y'
to it but when the variable 'y' is present in the code (even if not
present in the locals()), exec() does not add any 'y' (and the next
eval() then fails)

The interesting part is that if the 'y' variable is in the f's code
*and* it is defined in the f's locals, no error occur but once again the
exec() does not modify f's locals:

# test 3
def f():
     i = 1
     y = 42
     print(locals())
     exec('y = i; print(y); print(locals())')
     print(locals())
     a = eval('y')
     print(locals())
     y = a
     print(y)
f()
{'i': 1, 'y': 42}
1
{'i': 1, 'y': 1}
{'i': 1, 'y': 42}
{'i': 1, 'y': 42, 'a': 42}
42

Why does this happen? No idea.

I may be related with this:

# test 4
def f():
     i = 1
     print(locals())
     exec('y = i; print(y); print(locals())')
     print(locals())
     print(y)
f()
Traceback (most recent call last):
NameError: name 'y' is not defined

Despite exec() adds the 'y' variable to f's locals, the variable is not
accessible/visible from f's code.

So, a few observations (by no means this is how the vm works):

1) each function has a set of variables defined by the code (let's call
this "code-defined locals" or "cdef-locals").
2) each function also has a set of "runtime locals" accessible from
locals().
3) exec() can add variables to locals() (runtime) set but it cannot add
any to cdef-locals.
4) locals() may be a superset of cdef-locals (but entries in cdef-locals
which value is still undefined are not shown in locals())
5) due rule 4, exec() cannot add a variable to locals() if it is already
  present in the in cdef-locals.
6) when eval() runs, it uses locals() set for lookup

Perhaps rule 5 is to prevent exec() to modify any arbitrary variable of
the caller...

Anyways, nice food for our brains.

On Wed, Jul 20, 2022 at 04:56:02PM +0000, george trojan wrote:
>I wish I could understand the following behaviour:
>
>1. This works as I expect it to work:
>
>def f():
>    i = 1
>    print(locals())
>    exec('y = i; print(y); print(locals())')
>    print(locals())
>    exec('y *= 2')
>    print('ok:', eval('y'))
>f()
>
>{'i': 1}
>1
>{'i': 1, 'y': 1}
>{'i': 1, 'y': 1}
>ok: 2
>
>2. I can access the value of y with eval() too:
>
>def f():
>    i = 1
>    print(locals())
>    exec('y = i; print(y); print(locals())')
>    print(locals())
>    u = eval('y')
>    print(u)
>f()
>
>{'i': 1}
>1
>{'i': 1, 'y': 1}
>{'i': 1, 'y': 1}
>1
>
>3. When I change variable name u -> y, somehow locals() in the body of
>the function loses an entry:
>
>def f():
>    i = 1
>    print(locals())
>    exec('y = i; print(y); print(locals())')
>    print(locals())
>    y = eval('y')
>    print(y)
>f()
>
>{'i': 1}
>1
>{'i': 1, 'y': 1}
>{'i': 1}
>
>---------------------------------------------------------------------------NameError
>                                Traceback (most recent call last)
>Input In [1], in <cell line: 10>()      7     print(y)      8     # y
>= eval('y')      9     #print('ok:', eval('y'))---> 10 f()
>
>Input In [1], in f()      4 exec('y = i; print(y); print(locals())')
>   5 print(locals())----> 6 y = eval('y')      7 print(y)
>
>File <string>:1, in <module>
>NameError: name 'y' is not defined1.
>
>Another thing: within the first exec(), the print order seems
>reversed. What is going on?
>
>BTW, I am using python 3.8.13.
>
>George
>-- 
>https://mail.python.org/mailman/listinfo/python-list