Continuations and threads (was Re: Iterators & generators)

[Christian Tismer]

Tim Peters wrote:
> 
> [Thomas Wouters]
> > I may be too confused to tell for sure, but i have a gut feeling
> > that i'm getting closer and closer to some complete and utterly
> > consistent cluelessness of this issue.
> 
> That means you're close to getting frustrated enough to make a real
> breakthrough <0.9 wink>.

That's the way, I guess. At least it was mine,
and I'm known as the "Clueless Implementor" (TM).

> > ...
> > and this continuations business isn't some huge PSU conspiracy to
> > scare the bejezes out of everyone, to make sure continuations do
> > _not_ make it into Python ;)
> 
> That's indeed how this topic has ended each time over the years, and for
> exactly that reason.

The never-ending story will find an end. After I've found a way
to redefine truth, I will also find a redefinition of "never".

"never" is the female formulation of "perhaps", which is btw.
also applicable to Guido. (duck and cover)

> I was going to yell at you to read the docs, but after downloading all
> Christian's latest stuff I realized what should have been obvious before:
> while Christian has written a lot about the implementation, there are no
> docs explaining what the *user* sees -- there's not even a docstring to give
> you a hint about what continuation.caller etc do.  [Christian, that's why I
> copied you:  you're supposed to feel guilty now and stay up all weekend
> <wink>.]

Yessir. I felt guilty and added doc strings to everything I could find.
This doesn't make things easier, but it is now much easier to find
that out. :-)

I was not sure about my continuation stuff, and I was waiting for
some replies from the python-dev gurus. I thought they would read my
paper with their given understanding of all the co-stuff, and given that
everything should be obvious.
Now this dodn't work out. Quite a few dev-people did understand me,
without giving me feedback beyond "great work, but what now?".
Instead, I get lots of feedback from the "outer" world, and people
seem to really want to try this stuff, even when they have no clue
about anything that is non-procedureal.
I have to change some policies :-)

> So, for now, if you want to learn anything about how to use them, you're
> better off following one of the links Christian has posted into the Scheme
> world.

*Blush*, still true. But SLP 1.1 is about to be shipped,
and there are docstrings, and there is a *real nice generator*
now. Next time I will publish a how-to: How to get mad with
continuations, compared to other ways of becoming unhappy.

> OTOH, why do you care?  You're not intended to use them directly -- the only
> people who have any reason to figure this out are wishful implementers of
> high-level abstractions (like coroutines etc).
> 
> > ...
> > I'm probably looking at this all wrong -- i fail to see how this
> > is more abstract, more theoretical, more low-level, than simple
> > subroutines.
> 
> It's "simpler" in somewhat the same formal sense that it's simpler to define
> how to add 1 to an unbounded int than to define an algorithm for unbounded
> int multiplication.  Continuations are basic because all other known forms
> of control-flow can be reduced to them.  Simpler does *not* imply "easier"
> or "more convenient" here -- it does imply "low level", "tedious",
> "unfamiliar" and "conducive to cognitive dissonance" <0.9 wink>.  And, of
> course, there's nothing at all abstract about the implementation.  Separate
> the theory from the implementation -- there's nothing simple about the
> implementation!

Sure. Sigh. The implementation is a nightmare, partially.
But at the moment it's turning into a daymare, after I've
understood what stacks and interpreters are :-)
SLP 1.1 is much simpler and much more powerful. You can now
run any frame at any time, even from a different thread.
And the code has become shorter.
Not to mention that it is also faster than ever before:
7% more pystone than standard Python, although I have the
safe destructors of nested structures in it. Ahem :-)

> > Is there any chance that someone can explain continuations either
> > in a very pythonic sense
> 
> No, not at the language level:  they're akin to gotos, and Python has
> nothing like that today.  At the implementation level, though, see
> Christian's paper for the best explanation you're going to get.

Unfortunately, this paper is now *INVALID*.
I have to do a major rewrite.
Readers, please forget about the chapter concerning frame
compatibility. THIS IS NOT TRUE.
Please, also forget about dispatcher objects. THEY ARE GONE.

Sorry about shouting. This is more like shouting at myself,
since I was so dumb during all those months. Well, the
fortune is that I still can learn...

The author of this paper had no clue what he was talking
about, and he was trying to convince people of his thoughtless
conclusions. Well, not thoughtless (he spent months in despare,
like a Cyber 175 peripheral processor's "hung in automatic recall")
but clueless, no insight, not thinking consequently.

Yesterday I made it. Aleph One, everything is crystal clear.
I shortened the code by about 20 percent, and it is now faster,
more powerful, and "correct" if that is possible at all.

> > ....
> > Can't someone do a very simple example using toy cars, road crossings,
> > chickens, idiot drivers and/or huge kerosine-carrying tankertrucks ?
> > Something that might not be much clearer but sheds some very different
> ? light on the issue, and caters to the simple minds ? ;P
> 
> OK, I'll give a very simple example.  You're not going to like it -- in
> fact, you're going to hate it <wink>.  Like adding 1 to a number when you
> really want to multiply, it's not of obvious use, but illustrates
> fundamental points without the mastering of which you won't make progress.

The Stackless Python Homepage will become updated soon, including
such nice little hateful examples, *and* useful ones that make
unPythonic continuation hackery into nice and useful Pythonic
control structures.

God am I happy that things work so smoothly now :-)

> raw-continuations-are-the-antithesis-of-pythonic-ly y'rs  - tim

and-I'm-the-tricky-code-antichrist-who-never-gives-up-ly y'rs - chris

-- 
Christian Tismer             :^)   <mailto:tismer at appliedbiometrics.com>
Applied Biometrics GmbH      :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     we're tired of banana software - shipped green, ripens at home