[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

[STINNER Victor]

STINNER Victor <vstinner at python.org> added the comment:

Guido: "I'm still worried about the change in semantics where globals["__builtins__"] is assigned a different dict after the function object has been created (...)"

Well, there is a semantics change of Python 3.10 documented at:
https://docs.python.org/dev/whatsnew/3.10.html#other-language-changes

"Functions have a new __builtins__ attribute which is used to look for builtin symbols when a function is executed, instead of looking into __globals__['__builtins__']. (Contributed by Mark Shannon in bpo-42990.)"

And the function __builtins__ attribute is read-only.


Your example is not affected by PR 24564 because the globals has the "__builtins__" key.


In Python 3.10, you can modify func.__builtins__ (new attribute):
---
def foo(s): return len(s)
code = foo.__code__
FunctionType = type(foo)
f = FunctionType(code, {"__builtins__": {"len": len}})
print(f("abc"))
f.__builtins__.clear()
print(f("abc"))
---

Output:
---
3
Traceback (most recent call last):
  (...)
NameError: name 'len' is not defined
---


Mark: "Because globals['__builtins__'] is cached for each function activation, executing functions don't see updates."

In Python 3.10, if someone wants to hack builtins while the function is running, modifying the builtins namespace in-place works as expected:
---
def f():
    print(len("test"))
    builtins_ns = f.__globals__['__builtins__'].__dict__
    #builtins_ns = f.__builtins__
    builtins_ns['len'] = lambda x: 7
    print(len("test"))

f()
---

Output:
---
4
7
---

It also works with "builtins_ns = f.__builtins__".


Guido: "I realize this is a pretty esoteric, but it does show the change in semantics (from later to earlier binding). Should we care? I like early binding because it allows more optimizations[1], but traditionally Python's semantics use late binding."

Modifying built-in functions/types is commonly done in tests. Example:
---
import unittest.mock

def func():
    with unittest.mock.patch('builtins.chr', return_value='mock'):
        return chr(65)

print(func())
---

The expected output is: "mock". Overriding an attribute of the builtins module immediately updates func.__builtins__. It works because func.__builtins__ is builtins.__dict__.

In FAT Python, I implemented an optimization which copies builtin functions to constants, replace LOAD_GLOBAL with LOAD_CONST:
https://fatoptimizer.readthedocs.io/en/latest/optimizations.html#copy-builtin-to-constant

This optimization breaks this Python semantics, it is no longer possible to override builtin functions in tests:
https://fatoptimizer.readthedocs.io/en/latest/semantics.html#builtin-functions-replaced-in-the-middle-of-a-function

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42990>
_______________________________________