[Python-Dev] 'stackless' python?

[rushing at nightmare.com]

Guido van Rossum writes:
  > I've come up with two partial solutions: (1) allow for a way to
  > arrange for a call to be made immediately after you return to the
  > VM from C; this would take care of apply() at least and a few
  > other "tail-recursive" cases; (2) invoke a new VM when C code
  > needs a Python result, requiring it to return.  The latter clearly
  > breaks certain uses of coroutines but could probably be made to
  > work most of the time.  Typical use of the 80-20 rule.

I know this is disgusting, but could setjmp/longjmp 'automagically'
force a 'recursive call' to jump back into the top-level loop?  This
would put some serious restraint on what C called from Python could
do...

I think just about any Scheme implementation has to solve this same
problem... I'll dig through my collection of them for ideas.

  > In general, I still think it's a cool idea, but I also still think
  > that continuations are too complicated for most programmers.  (This
  > comes from the realization that they are too complicated for me!)
  > Corollary: even if we had continuations, I'm not sure if this would
  > take away the resistance against asyncore/asynchat.  Of course I could 
  > be wrong.

Theoretically, you could have a bit of code that looked just like
'normal' imperative code, that would actually be entering and exiting
the context for non-blocking i/o.  If it were done right, the same
exact code might even run under 'normal' threads.

Recently I've written an async server that needed to talk to several
other RPC servers, and a mysql server.  Pseudo-example, with
possibly-async calls in UPPERCASE:

  auth, archive = db.FETCH_USER_INFO (user)
  if verify_login(user,auth):
    rpc_server = self.archive_servers[archive]
    group_info = rpc_server.FETCH_GROUP_INFO (group)
    if valid (group_info):
      return rpc_server.FETCH_MESSAGE (message_number)
    else:
      ...
   else:
     ...

This code in CPS is a horrible, complicated mess, it takes something
like 8 callback methods, variables and exceptions have to be passed
around in 'continuation' objects.  It's hairy because there are three
levels of callback state.  Ugh.

If Python had closures, then it would be a *little* easier, but would
still make the average Pythoneer swoon.  Closures would let you put
the above logic all in one method, but the code would still be
'inside-out'.

  > Different suggestion: it would be cool to work on completely
  > separating out the VM from the rest of Python, through some kind of
  > C-level API specification.

I think this is a great idea.  I've been staring at python bytecodes a
bit lately thinking about how to do something like this, for some
subset of Python.

[...]

Ok, we've all seen the 'stick'.  I guess I should give an example of
the 'carrot': I think that a web server built on such a Python could
have the performance/scalability of thttpd, with the
ease-of-programming of Roxen.  As far as I know, there's nothing like
it out there.  Medusa would be put out to pasture. 8^)

-Sam