[issue40077] Convert static types to PyType_FromSpec()

[STINNER Victor]

STINNER Victor <vstinner at python.org> added the comment:

In the _json module, PyModule_GetState() (called by get_json_state()) is only used by the garbage collector (traverse function) and to unload the module (clear and free functions). It's not used in "hot code" (let me consider that the GC is not part of the usual "hot code" :-)).

But maybe we should measure the overhead of future changes if PyModule_GetState() starts to be used in "hot code" by running a few microbenchmarks.

--

Stefan Krah:
> Yes, and not only startup time:
> https://bugs.python.org/issue15722

Aha, that's interesting. I didn't know that it could have an impact on runtime performance as well.

_decimal_pep3121-384_v1.patch attached to bpo-15722 seems to use:

#define _decimal_state_global ((_decimal_state *)PyModule_GetState(PyState_FindModule(&_decimal_module)))

whereas the commit 33f15a16d40cb8010a8c758952cbf88d7912ee2d only uses:

static inline _jsonmodulestate*
get_json_state(PyObject *module)
{
    void *state = PyModule_GetState(module);
    assert(state != NULL);
    return (_jsonmodulestate *)state;
}

Maybe PyState_FindModule() adds a little overhead, even if this function is simple: in short, it gets the i-th item of a list (from PyInterpreterState.modules_by_index).

PyModule_GetState() function is really simple: it only reads PyModuleObject.md_state attribute.

Or maybe _decimal_state_global was used in "hot code".

If PyState_FindModule() or PyModule_GetState() is the source of the overhead, maybe we could try to optimise these functions, or pass directly the module state to inner functions.

--

PyState_FindModule() doesn't work for a module using PyModuleDef_Init(). The PEP 573 is going to give access to the module state in functions which didn't access to it previsouly.

The commit 33f15a16d40cb8010a8c758952cbf88d7912ee2d removed a few assertions checking that "self" has the expected type. It wasn't possible to get the module state to get the types, because PEP 573 is not implemented yet and PyState_FindModule() doesn't work in _json (which uses PyModuleDef_Init()). I decided that it's ok to remove these assertions: it should not be possible to call these functions with another type in practice.

--

In his PR 19177, Dong-hee started by replacing direct access to PyTypeObject fields, like replacing:

    type->free(self);

with:

    freefunc free_func = PyType_GetSlot(tp, Py_tp_free);
    free_func(self);

I asked him to revert these changes.

I'm interested to experiment a few C extension modules of the stdlib using the limited C API (PEP 384, stable ABI), but I don't think that it should done while converting modules to PyType_FromSpec().

We should separate the two changes. And I would prefer to first see the overhead of PyType_FromSpec(), and discuss the advantages and drawbacks.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40077>
_______________________________________