Is Stackless Python DEAD?

[Christian Tismer]

Well, it is a little late to answer this, but...

Michael Hudson wrote:

 > Martin von Loewis <loewis at informatik.hu-berlin.de> writes:
...

...

 >>- It adds a field nesting_level to the thread state, without
 >>  ever checking its value (it just counts the nesting level)
 >>
 >
 > I imagine this was just for bragging with :)


Yes. I needed a way to track when and why there are still
recursions. At sme pahse I also had the idea to drive
decisions on this when I'm allowed to switch things, but
later I learned that this is the wrong way.

 >>- it adds a number of _nr variants of functions (non-recursive),
 >>  e.g. for map and eval. In __builtins__, the _nr versions are
 >>  available as "map" and "eval", while the original versions are
 >>  preserved as apply_orig and map_orig:
 >>  * Are the _nr versions functionally completely backwards-compatible?
 >>    If not, why? If yes, why is the original version preserved?
 >>
 >
 > I think the originals are just around because the implementation of
 > the _nr variants was tricky and Chris needed something to test
 > against.  Not sure, though.


A) they are completely compatible except for the special unwinding
rule. A special token may be returned that tells to unwind the stack.
The versions without _nr are kept for binary compatibility with
existing code which is not aware of the unwind token.
In this case, Stackless behaves like standard Python, it just
does recursions. The _nr versions are for extensions which
make use of the stackless features.

 >>  * Just to implement map, 150 lines of builtin_map had to be
 >>    rewritten into 350 lines (builtin_map, make_stub_code,
 >>    make_map_frame, builtin_map_nr, builtin_map_loop). The author
 >>    indicates that the same procedure still needs to be done for
 >>    apply and filter. Just what is the "same procedure"? Isn't there
 >>    some better way?
 >>
 >
 > This is where implementing stackless in C really, really hurts.


[great explanation of stackless techniques skipped.]

I agree this is not easy to undrstand and to implement.
I always was thinking of a framework which makes this
easier, but I didn'tcome up with something suitable.


...


 >>- The code adds PREPARE macros into each branch of ceval. Why?
 >>- It adds a long list of explicitly not-supported opcodes into
 >>  the ceval switch, instead of using 'default:'. No explanation
 >>  for that change is given, other than 'unused opcodes go here'.
 >>  Is it necessary to separately maintain them? Why?
 >>
 >
 > This was an optimization Chris used to try and get back some of the
 > performance lost during the stackless changes.  IIRC, he handles
 > exceptions and return values as "pseudo-opcodes" rather than using the
 > WHY_foo constants the current ceval.c uses.  I never really understood
 > this part.


This is really just an optimization.
The PREPARE macros were used to limit code increase, and to
gove me some more options to play with.
Finally, the PREPARE macros do an optimized opcode prefetch
which turns out to be a drastical speedup for the interpreter loop.
Standard Python does an increment for every byte code and then
one for the optional argument, and the argument is picked bytewise.
What I do is a single add to the program counter, dependent of the
opcode/argument size which is computed in the PREPARE macro.
Then, on intel machines, I use a short word access to the argument
which gives a considerable savings. (Although this wouldn't be
necessary if the compilers weren't that dumb).

 >>It may be that some of these questions can be answered giving a good
 >>reason for the change, but I doubt that this code can be incorporated
 >>as-is, just saying "you need all of this for Stackless Python". I
 >>don't believe you do, but I cannot work it out myself, either.
 >>
 >
 > I think integrating stackless into the core is a fairly huge amount of
 > work.  I'd like to think I could do it, given several months of
 > full-time effort (which isn't going to happen).  About the only likely
 > way I see for it to get in is for it to become important to Zope
 > Corp. for some reason, and them paying Tim or Guido (or Chris) to do
 > it.


I'm at a redesign for Stackless 2.2. I hope to make it simpler,
split apart Stackless and optimization, and continuations are
no longer my primary target, but built-in microthreads.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
       where do you want to jump today?   http://www.stackless.com/