[Python-3000] Practical uses of stackless technology

[Richard Tew]

On 10/23/06, Talin <talin at acm.org> wrote:
> Carl Friedrich Bolz wrote:
> >> Being technical hacker types, we can cope with describing the ins and
> >> outs of how our code works, but are less sure on the motivations for
> >> stackless-style technologies as used in real-world applications :)
> >> (especially since this part of PyPy is not used for any applications
> >> yet).
>
> Maybe the wrong list - but maybe the right discussion. Specifically, how
>   much of the stackless technology should be adopted in Python3K?

I currently maintain Stackless for what its worth and I also use it in
my day to day work at CCP.

The primary benefits of Stackless IMO are gained through the use of
the minimal but solid set of features which it provides, implemented on
top of its switching.

I am refering to the scheduler, the tasklets (aka microthreads) and
channels which it provides.  These are all tools which I consider a
natural enhancement to Python because of the way they facilitate
writing simple and readable code.

What I am saying is that the biggest gain is to adopt it in its
entirely.

> I understand, from reading the docs on stackless, that it jumps through
> some fairly significant hoops to try and keep the Python C API intact.
> Given the "cruft-removal" nature of Py3K, what could be done to minimize
> the amount of hoop-jumping that stackless has to do?

I think that any docs you may find on the web should be taken
with a grain of salt.  Pretty much all of them are out of date.  And
while there might be some aspects that are still relevant, I know
they only served to confuse me when I was initially trying to
understand Stackless so that I could port it to later releases of
mainline Python.  Stackless has changes a lot since it had
continuations at its core (nowadays the complication they
gave is nowhere to be seen and it has had several rewrites
since then IIRC).

As I understand it, Stackless does not do its hoop-jumping
to keep the C API intact.  Rather the smaller part of its
patches which could be qualified as hoop jumping serve
two main purposes.

 a) Unwinding and reducing how much of the stack is used.
 b) Switching to another tasklet without doing the "hard"
     switching. (otherwise known as "soft" switching).

"Hard" switching is where the stack contents are saved, and
the hand-written assembly code is invoked to change the
stack pointer and save any registers.  Reducing the how much
stack is used also of course reduces how much the "hard"
switching has to save.

The bulk of the Stackless code is in the Stackless
directory, and implements hard switching, the
scheduler, tasklets and channels.  And pickling of
running code of course.

The smaller part of it is the changes to mainline code.

Hope this helps,
Richard.