[Python-Dev] uthread strawman

[Christian Tismer]

Guido van Rossum wrote:
> 
> > Guido:
> >
> > > I hope that it will be possible to make it a lot simpler than current
> > > stackless though, by not doing continuations.
> 
> [Greg Ewing]
> > My feeling is that this won't be the case. The fundamental
> > change of structure needed to make it stackless will be
> > much the same, as will the thought processes necessary
> > to understand how it works.
> 
> I hope you are wrong but you may be right.  I'll have to have a good
> look -- or someone else (not Christian!  With all due respect his code
> is unreadable :-).

Are you talking of my changes to ceval.c or the continuationmodule?
I think it can't be the latter, since that does not matter at all
if we talk about Stackless. Stackless was written to make continuations
possible. It does not implement them.

My changes to ceval.c are written in the same style as your
original code, and it uses the same level of commenting as yours:
Nothing at all. :-)
With all due respect, I consider both versions equally unreadable,
unless one understands what the intent of the code is.

Until last October, I tried to keep everything as readable
and understandable as possible. Then it became clear that
this implementation would never make it into the core.
Then I gave up my efforts, and I also added a lot of
optimizations to the interpreter, by some systematic use
of macroes. Shurely his doesn't increase readability.

Forgetting about these optimizations, the code doesn't do
much more than the following:

eval_code2_setup
  is split off of the original eval_code2 function.
  It prepares a new frame for execution and puts it
  on top of the frame stack.

PyEval_Frame_Dispatch
  is a new function. It controls the execution of frames.
  Every initial or recursive interpreter call starts
  such a dispatcher. The nested dispatchers control the
  remaining "stackful" execution of Python.
  In its central loop, it runs the topmost frame of the
  frame stack, receives its return value and runs the
  next frame, until it sees the frame appear that
  spawned this dispatcher. Then it returns.

eval_code2_loop
  is the "real" part of the original eval_code2 function.
  It is not much different from the original. Major changes
  have been done to the entry code, the periodic checks
  in the loop, and the handling of function calls.

  The "big switch" has been simplified in the sense, that
  errors are no longer treated with various variables
  which have to be checked outside the switch.
  Instead, errors are directly mapped on a pseudo-opcode
  that allows to handle exceptions as just another case
  of the big switch.

  Every function call has got additional code that checks
  for the so-called unwind token, which tells us to leave
  this frame and to return to the scheduler.
  
  On entry to the frame, on every trip through the main loop,
  and after every function call, a callback f_callguard is
  checked for existence. If it exists, it is called, and
  if it returns -42, again the frame is left and we return
  to the scheduler.

  Entry into a frame has become a bit difficult, since we
  no longer know in advance whether a frame is expected
  to return a value or not. Due to uthread scheduling,
  switches occour between opcodes, and no values are
  transferred. When switching in the context of a function
  call, there *are* return values expected.
  This is all handled via some flags, in the frame entry
  code, line 948ff.

  Then, there are some very simple changes to the loop
  construct.

  Generally, more state variables are in the frames and
  kept up-to-date, like the instruction pointer.

  I'm omitting the extra code for uthread support here.

  Some functions were pulled out of the main loop, in order
  to make it smaller and easier to read. I would undo this
  today, since it makes comparison to the old version
  quite impossible, and it didn't yield more speed.

This is about all of it.
As you can see, there is no explicit support for co-anything
in the code. There are some generalisations to frame calling
and some callback hooks which actually do all co operations.

An implementation targeted for core integration would look
quite much different. It would provide more functionality
directly, without using callbacks.

A pure coroutine based implementation as you proposed
would not need the generalization of the frame parameter
passing, since switches can only occour in the context
of a function call.

Supporting auto-scheduled uthreads needs to distinguish
explicit and implicit switching, since implicit switching
occours between opcodes, not *in* opcodes.

The techniques for this can be written in quite a different
manner than I did.

Again, this code is not intended for inclusion in the core,
and not for drawing conclusions for the feasibility of
Stackless at all. The latter has been shown by applications
like the uthreads, and by its central use in the EVE game.
We need to judge the priciple, not the implementaiton.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com