[Python-Dev] Stackless Python - Pros and Cons

[Jeremy Hylton]

If someone is going to write a PEP, I hope they will explain how the
implementation deals with the various Python C API calls that can call back
into Python.

In the stackless implementation, builtin_apply is a thin wrapper around
builtin_apply_nr.  The wrapper checks the return value from builtin_apply_nr
for Py_UnwindToken.  If Py_UnwindToken is found, it calls
PyEval_Frame_Dispatch. In this case, builtin_apply returns whatever
PyEval_Frame_Dispatch returns; the frame dispatcher just executes stack
frames until it is ready to return.

How does this control flow at the C level interact with a Python API call
like PySequence_Tuple or PyObject_Compare that can start executing Python
code again?  Say there is a Python function call which in turn calls
PySequence_Tuple, which in turn calls a __getitem__ method on some Python
object, which in turn uses a continuation to transfer control.  After the
continuation is called, the Python function will never return and the
PySquence_Tuple call is no longer necessary, but there is still a call to
PySequence_Tuple on the C stack.  How does stackless deal with the return
through this function?

I expect that any C function that may cause Python code to be executed must
be wrapped the way apply was wrapper.  So in the example, PySequence_Tuple
may return Py_UnwindToken.  This adds an extra return condition that every
caller of PySequence_Tuple must check.  Currently, the caller must check for
NULL/exception in addition to a normal return.  With stackless, I assume the
caller would also need to check for "unwinding."

Is this analysis correct? Or is there something I'm missing?

I see that the current source release of stackless does not do anything
special to deal with C API calls that execute Python code.  For example,
PyDict_GetItem calls PyObject_Hash, which could in theory lead to a call on
a continuation, but neither caller nor callee does anything special to
account for the possibility.  Is there some other part of the implementation
that prevents this from being a problem?

Jeremy