[New-bugs-announce] [issue43541] PyEval_EvalCodeEx() can no longer be called with code which has (CO_NEWLOCALS | CO_OPTIMIZED) flags

[STINNER Victor]

New submission from STINNER Victor <vstinner at python.org>:

Cython generates a __Pyx_PyFunction_FastCallDict() function which calls PyEval_EvalCodeEx(). With Python 3.9, it worked well. With Python 3.10 in debug mode, it fails with an assertion error:

python3.10: Python/ceval.c:5148: PyEval_EvalCodeEx: Assertion `(((PyCodeObject *)_co)->co_flags & (CO_NEWLOCALS | CO_OPTIMIZED)) == 0' failed.

With Python 3.10 in release mode, it does crash.

Context of the failed assertion:

* Assertion added recently to CPython 3.10 by python/cpython at 0332e56
* The code object flags = (CO_NEWLOCALS | CO_OPTIMIZED | CO_NOFREE)
* Code co_argcount = 2
* Code co_kwonlyargcount = 0
* Cython __Pyx_PyFunction_FastCallDict() called with: nargs=1 and kwargs=NULL

See the Cython issue to a reproducer: https://github.com/cython/cython/issues/4025#issuecomment-801829541

In Python 3.9, _PyFunction_Vectorcall() has the following fast-path:

    if (co->co_kwonlyargcount == 0 && nkwargs == 0 &&
        (co->co_flags & ~PyCF_MASK) == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE))
    {
        if (argdefs == NULL && co->co_argcount == nargs) {
            return function_code_fastcall(tstate, co, stack, nargs, globals);
        }
        else if (nargs == 0 && argdefs != NULL
                 && co->co_argcount == PyTuple_GET_SIZE(argdefs)) {
            /* function called with no arguments, but all parameters have
               a default value: use default values as arguments .*/
            stack = _PyTuple_ITEMS(argdefs);
            return function_code_fastcall(tstate, co,
                                          stack, PyTuple_GET_SIZE(argdefs),
                                          globals);
        }
    }

When the bug occurs, __Pyx_PyFunction_FastCallDict() doesn't take the fast-path because nargs < co_argcount (1 < 2).

In Python 3.10, _PyFunction_Vectorcall() is very different:

    if (((PyCodeObject *)f->fc_code)->co_flags & CO_OPTIMIZED) {
        return _PyEval_Vector(tstate, f, NULL, stack, nargs, kwnames);
    }
    else {
        return _PyEval_Vector(tstate, f, f->fc_globals, stack, nargs, kwnames);
    }


PyEval_EvalCodeEx() must not crash if the code object has (CO_NEWLOCALS | CO_OPTIMIZED | CO_NOFREE) flags.

----------
components: C API
messages: 389009
nosy: Mark.Shannon, pablogsal, vstinner
priority: normal
severity: normal
status: open
title: PyEval_EvalCodeEx() can no longer be called with code which has (CO_NEWLOCALS | CO_OPTIMIZED) flags
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43541>
_______________________________________