[issue43760] The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)
STINNER Victor
report at bugs.python.org
Fri Sep 24 19:12:48 EDT 2021
STINNER Victor <vstinner at python.org> added the comment:
Analysis use use_tracing usage in 3rd part code.
I see two main ways to add C API functions covering these use cases:
* Provide high-level functions like "call a trace function" (disable tracing, call trace function, reenable tracing, increment/decrement tstate->tracing)
* Provide low-level functions just to control use_tracing: make PyThreadState structure opaque, but stil make the assumption that it is possible to disable temporarily tracing and profiling (in practice, it's implemented as use_tracing=0).
(*) greenlet
greenlet disables temporarily tracing in g_calltrace(), and then restore it, to call a "tracing" function:
---
tstate->tracing++;
TSTATE_USE_TRACING(tstate) = 0;
retval = PyObject_CallFunction(tracefunc, "O(OO)", event, origin, target);
tstate->tracing--;
TSTATE_USE_TRACING(tstate) =
(tstate->tracing <= 0 &&
((tstate->c_tracefunc != NULL) || (tstate->c_profilefunc != NULL)));
---
It also saves and then restores use_tracing value:
---
ts__g_switchstack_use_tracing = tstate->cframe->use_tracing;
(...)
tstate->cframe->use_tracing = ts__g_switchstack_use_tracing;
---
=> it can use PyThreadState_IsTracing(), PyThreadState_DisableTracing() and PyThreadState_ResetTracing().
These functions don't handle "tstate->tracing++;" and "tstate->tracing--;" which is also used by greenlet.
greenlet also saves and restores tstate->cframe:
https://github.com/python-greenlet/greenlet/blob/master/src/greenlet/greenlet.c
(*) dipy
Code generated by Cython.
(*) smartcols
Code generated by Cython.
(*) yappi
yappi is Python profiler.
yappi sets use_tracing to 1 when it sets its profile function: "ts->c_profilefunc = _yapp_callback;".
It sets use_tracing to 0 when it clears the profile function: "ts->c_profilefunc = NULL;". That's wrong, it ignores the trace function.
PyEval_SetProfile() cannot be used because yappi works on a PyThreadState (ts).
Code: https://github.com/sumerc/yappi/blob/master/yappi/_yappi.c
It can use PyThreadState_DisableTracing() and PyThreadState_ResetTracing(). Maybe a PyThreadState_SetProfile(tstate, func) function would fit better yappi's use case.
(*) Cython
Cython defines 2 compatibility functions:
* __Pyx_IsTracing(tstate, check_tracing, check_funcs): it can check c_profilefunc and c_tracefunc
* __Pyx_SetTracing(tstate, enable)
Code: https://github.com/cython/cython/blob/0.29.x/Cython/Utility/Profile.c
The code is quite complicated. In short, it checks if tracing and/or profiling is enabled. If it's enabled, it disables temporarily tracing (use_tracing=0) while calling trace and profile functions.
=> it requires PyThreadState_IsTracing(), PyThreadState_DisableTracing() and PyThreadState_ResetTracing().
----------
_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43760>
_______________________________________
More information about the Python-bugs-list
mailing list