Fake threads (was [Python-Dev] ActiveState & fork & Perl)

[Tim Peters]

[Gordon McMillan]
> Hmmm. I jumped back into this one, but never saw my post show up...

Me neither!  An exclamation point because I see there's a recent post of
yours in the Python-Dev archives, but I didn't get it in the mail either.

> Threads (real or fake) are useful when more than one thing is
> "driving" your processing. It's just that in the real world (a place
> Tim visited, once, but didn't like - or was it vice versa?) those
> "drivers" are normally I/O.

Yes, but that's the consensus view of "real", and so suffers from "ten
billion flies can't be wrong" syndrome <wink>.  If you pitch a parallel
system to the NSA, they give you a long list of problems and ask you to
sketch the best way to solve them on your platform; as I recall, none had
anything to do with I/O even under Guido's definition; instead tons of
computation with difficult twists, and enough tight coupling to make threads
the natural approach in most cases.  If I said any more they'd terminate me
with extreme prejudice, and the world doesn't get any realer than that
<wink>.

> Guido complained that to do it right would require gathering up all
> the fds and doing a select. I don't think that's true (at least, for
> a decent fake thread). You just have to select on the one (to see if
> the I/O will work) and swap or do it accordingly. Also makes it a bit
> easier for portability (I thought I heard that Mac's select is
> limited to sockets).

Can you flesh out the "swap" part more?  That is, we're in the middle of
some C code, so the C stack is involved in the state that's being swapped,
and under fake threads we don't have a real thread to magically capture
that.

> I see 2 questions. First, is there enough of an audience (Mac,
> mostly, I think) without native threads to make them worthwhile?
> Second, do we want to introduce yet more possibilities for
> brain-explosions by enabling coroutines / continuations / generators
> or some such? There is practical value there (as Sam has pointed out,
> and I now concur, watching my C state machine grow out of control
> with each new client request).
>
> I think the answer to both is probably "yes", and though they have a
> lot in common technically, they have totally different rationales.

a) Generators aren't enough for Sam's designs.

b) Fake threads are roughly comparable to coroutines and continuations wrt
power (depending on implementation details, continuations may be strictly
most powerful, and coroutines least).

c) Christian's stackless Python can, I believe, already do full coroutines,
and is close to doing full continuations.  So soon we can kick the tires
instead of each other <wink>.

or-what-the-heck-we-can-akk-kick-chris-ly y'rs  - tim