[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)

[STINNER Victor]

STINNER Victor <vstinner at python.org> added the comment:

Analysis use use_tracing usage in 3rd part code.

I see two main ways to add C API functions covering these use cases:

* Provide high-level functions like "call a trace function" (disable tracing, call trace function, reenable tracing, increment/decrement tstate->tracing)
* Provide low-level functions just to control use_tracing: make PyThreadState structure opaque, but stil make the assumption that it is possible to disable temporarily tracing and profiling (in practice, it's implemented as use_tracing=0).



(*) greenlet

greenlet disables temporarily tracing in g_calltrace(), and then restore it, to call a "tracing" function:
---
    tstate->tracing++;
    TSTATE_USE_TRACING(tstate) = 0;
    retval = PyObject_CallFunction(tracefunc, "O(OO)", event, origin, target);
    tstate->tracing--;
    TSTATE_USE_TRACING(tstate) =
        (tstate->tracing <= 0 &&
         ((tstate->c_tracefunc != NULL) || (tstate->c_profilefunc != NULL)));
---

It also saves and then restores use_tracing value:
---
ts__g_switchstack_use_tracing = tstate->cframe->use_tracing;
(...)
tstate->cframe->use_tracing = ts__g_switchstack_use_tracing;
---

=> it can use PyThreadState_IsTracing(), PyThreadState_DisableTracing() and PyThreadState_ResetTracing().

These functions don't handle "tstate->tracing++;" and "tstate->tracing--;" which is also used by greenlet.

greenlet also saves and restores tstate->cframe:
https://github.com/python-greenlet/greenlet/blob/master/src/greenlet/greenlet.c


(*) dipy

Code generated by Cython.


(*) smartcols

Code generated by Cython.


(*) yappi

yappi is Python profiler.

yappi sets use_tracing to 1 when it sets its profile function: "ts->c_profilefunc = _yapp_callback;".

It sets use_tracing to 0 when it clears the profile function: "ts->c_profilefunc = NULL;". That's wrong, it ignores the trace function.

PyEval_SetProfile() cannot be used because yappi works on a PyThreadState (ts).

Code: https://github.com/sumerc/yappi/blob/master/yappi/_yappi.c

It can use PyThreadState_DisableTracing() and PyThreadState_ResetTracing(). Maybe a PyThreadState_SetProfile(tstate, func) function would fit better yappi's use case.


(*) Cython

Cython defines 2 compatibility functions:

* __Pyx_IsTracing(tstate, check_tracing, check_funcs): it can check c_profilefunc and c_tracefunc
* __Pyx_SetTracing(tstate, enable)

Code: https://github.com/cython/cython/blob/0.29.x/Cython/Utility/Profile.c

The code is quite complicated. In short, it checks if tracing and/or profiling is enabled. If it's enabled, it disables temporarily tracing (use_tracing=0) while calling trace and profile functions.

=> it requires PyThreadState_IsTracing(), PyThreadState_DisableTracing() and PyThreadState_ResetTracing().

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43760>
_______________________________________