Fake threads (was [Python-Dev] ActiveState & fork & Perl)

[Christian Tismer]

Tim Peters wrote:
> 
> [Mark Hammond]
> > ...
> > Thanks for the considerable time it must be taking to enlightening us!
> 
> You're welcome, but the holiday weekend is up and so is my time.  Thank (all
> of) *you* for the considerable time it must take to endure all this <wink>!

Just to let you know that I'm still there, thinking, not coding,
still hesitating, but maybe I can conclude now and send it off.

This discussion, and especially Tim's input was extremely helpful.
He has spent considerable time reading my twisted examples,
writing his own, hitting my chin, kicking my -censored-, and
proving to me that the truth I was searching doesn't exist.

...
> Again, though, you never want to muck with continuations directly!  They're
> too wild.  You get an expert to use them in the bowels of an implementation
> of something else.

Maybe with one exception: With careful coding, you can use
a continuation at the head of a very deep recursion and use
it as an early break if the algorithm fails. The effect is
the same as bailing out with an exception, despite the fact
that no "finally" causes would be obeyed. It is just a
incredibly fast jump out of something if you know what
you are doing.

> > With continuations, how is the state captured or created?
> 
> There are, of course, many ways to implement these things.  Christian is
> building them on top of the explicit frame objects Python already creates,
> and that's a fine way for Python.  Guido views it as cloning the call stack,
> and that's accurate too.

Actually, it is both!
What I use (and it works fine) are so-called "push-back frames".
My continuations are always appearing in some call.
In order to make the caller able to be resumed, I create
a push-back frame *from* it. That means, my caller frame is
duplicated behind his "f_back" pointer. The original frame
stays in place but now becomes a continuation frame with
the current stack state preserved. All other locals and stuff
are moved to the clone in the f_back which is now the real one.
This always works fine, since references to the original
caller frame are all intact, just the frame's meaning is
modified a little.

Well, I will hvae to write a good paper...

... 

> I personally have wanted generators in Python since '91, because they're
> extremely useful in the useless things that I do <wink>.  There's a
> thread-based generator interface (Generator.py) in the source distribution
> that I occasionally use, but that's so slow I usually recode in Icon (no,
> I'm not a Scheme fan -- I *admire* it, but I rarely use it).  I expect
> rebuilding that on Christian's work will yield a factor of 10-100 speedup
> for me (beyond losing the thread/mutex overhead, as Chris just pointed out
> on c.l.py resumption should be much faster than a Python call, since the
> frame is already set up and raring to go).

I believe so. Well, I admit that the continuation approach is
slightly too much for the coroutine/generator case, since they
exactly don't have the problem where continuations are suffering
a little: Switching between frames which cannot be reached more
than once at a time don't need the stack copying/pushback at all.
I'm still staying at the secure side for now. But since I have
all refcounting accurate already, we can use it to figure out if
a frame needs to be copied at all.

> Would be nice if the language grew some syntax to make generators pleasant
> as well as fast, but the (lack of) speed is what's really killing it for me
> now.

How about "amb"? :-)
(see "teach youself schem in fixnum days, chapter 14 at
http://www.cs.rice.edu/~dorai/t-y-scheme/t-y-scheme-Z-H-15.html#%_chap_14)

About my last problems:
The hard decision is:
- Either I just stop and I'm ready already, and loops are funny.
- Or I do the hidden register search, which makes things more
  complicated and also voidens the pushback trick partially,
  since then I would manage all stack stuff in one frame.
- Or, and that's what I will do finally:
  For now, I will really just correct the loops.

Well, that *is* a change to Python again, but no semantic change.
The internal loop counter will no longer be an integer object,
but a mutable integer box. I will just create a one-element
integer array and count with its zero element.
This is correct, since the stack value isn't popped off,
so all alive stack copies share this one element.

As a side effect, I save the Object/Integer conversion, so
I guess it will be faster. *and* this solution does not involve
any other change, since the stack layout is identical to before.

-- 
Christian Tismer             :^)   <mailto:tismer@appliedbiometrics.com>
Applied Biometrics GmbH      :     Have a break! Take a ride on Python's
Kaiserin-Augusta-Allee 101   :    *Starship* http://starship.python.net
10553 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     we're tired of banana software - shipped green, ripens at home