[Python-Dev] 'stackless' python?

[Tim Peters]

[Christian]
[... clarified stuff ... thanks! ... much clearer ...]
> ...
> If a frame is the current state, I make it two frames to have two
> current states.  One will be saved, the other will be run. This is
> what I call "splitting".  Actually, splitting must occour whenever
> a frame can be reached twice, in order to keep elements alive.

That part doesn't compute:  if a frame can be reached by more than one path,
its refcount must be at least equal to the number of its immediate
predecessors, and its refcount won't fall to 0 before it becomes
unreachable.  So while you may need to split stuff for *some* reasons, I
can't see how keeping elements alive could be one of those reasons (unless
you're zapping frame contents *before* the frame itself is garbage?).

> ...
> Well, I see. You want one locals and one globals, shared by two
> incarnations. Gets me into trouble.

Just clarifying what Scheme does.  Since they've been doing this forever, I
don't want to toss their semantics on a whim <wink>.  It's at least a
conceptual thing:  why *should* locals follow different rules than globals?
If Python2 grows lexical closures, the only thing special about today's
"locals" is that they happen to be the first guys found on the search path.
Conceptually, that's really all they are today too.

Here's the clearest Scheme example I can dream up:

(define k #f)

(define (printi i)
  (display "i is ") (display i) (newline))

(define (test n)
  (let ((i n))
    (printi i)
    (set! i (- i 1))
    (printi i)
    (display "saving continuation") (newline)
    (call/cc (lambda (here) (set! k here)))
    (set! i (- i 1))
    (printi i)
    (set! i (- i 1))
    (printi i)))

No loops, no recursive calls, just a straight chain of fiddle-a-local ops.
Here's some output:

> (test 5)
i is 5
i is 4
saving continuation
i is 3
i is 2
> (k #f)
i is 1
i is 0
> (k #f)
i is -1
i is -2
> (k #f)
i is -3
i is -4
>

So there's no question about what Scheme thinks is proper behavior here.

> ...
> Let me explain. What Python does right now is:
> When a function is invoked, all local variables are copied
> into fast_locals, well of course just references are copied
> and counts increased. These fast locals give a lot of speed
> today, we must have them.

Scheme (most of 'em, anyway) also resolves locals via straight base + offset
indexing.

> You are saying I have to share locals between frames. Besides
> that will be a reasonable slowdown, since an extra structure
> must be built and accessed indirectly (right now, i's all fast,
> living in the one frame buffer),

GETLOCAL and SETLOCAL simply index off of the fastlocals pointer; it doesn't
care where that points *to* <wink -- but, really, it could point into some
other frame and ceval2 wouldn't know the difference).  Maybe a frame entered
due to continuation needs extra setup work?  Scheme saves itself by putting
name-resolution and continuation info into different structures; to mimic
the semantics, Python would need to get the same end effect.

> I cannot say that I'm convinced that this is what we need.
>
> Suppose you have a function
>
> def f(x):
>     # do something
>     ...
>     # in some context, wanna have a snapshot
>     global snapshot  # initialized to None
>     if not snapshot:
>         snapshot = callcc.new()
>     # continue computation
>     x = x+1
>     ...
>
> What I want to achieve is that I can run this again, from my
> snapshot. But with shared locals, my parameter x of the
> snapshot would have changed to x+1, which I don't find useful.

You need a completely fleshed-out example to score points here:  the use of
call/cc is subtle, hinging on details, and fragments ignore too much.  If
you do want the same x,

    commonx = x
    if not snapshot:
         # get the continuation
    # continue computation
    x = commonx
    x = x+1
    ...

That is, it's easy to get it.  But if you *do* want to see changes to the
locals (which is one way for those distinct continuation invocations to
*cooperate* in solving a task -- see below), but the implementation doesn't
allow for it, I don't know what you can do to worm around it short of making
x global too.  But then different *top* level invocations of f will stomp on
that shared global, so that's not a solution either.  Maybe forget functions
entirely and make everything a class method.

> I want to fix a state of the current frame and still think
> it should "own" its locals. Globals are borrowed, anyway.
> Class instances will anyway do what you want, since
> the local "self" is a mutable object.
>
> How do you want to keep computations independent
> when locals are shared? For me it's just easier to
> implement and also to think with the shallow copy.
> Otherwise, where is my private place?
> Open for becoming convinced, of course :-)

I imagine it comes up less often in Scheme because it has no loops:
communication among "iterations" is via function arguments or up-level
lexical vrbls.

So recall your uses of Icon generators instead:  like Python, Icon does have
loops, and two-level scoping, and I routinely build loopy Icon generators
that keep state in locals.  Here's a dirt-simple example I emailed to Sam
earlier this week:

procedure main()
    every result := fib(0, 1) \ 10 do
        write(result)
end

procedure fib(i, j)
    local temp
    repeat {
        suspend i
        temp := i + j
        i := j
        j := temp
    }
end

which prints

0
1
1
2
3
5
8
13
21
34

If Icon restored the locals (i, j, temp) upon each fib resumption, it would
generate a zero followed by an infinite sequence of ones(!).

Think of a continuation as a *paused* computation (which it is) rather than
an *independent* one (which it isn't <wink>), and I think it gets darned
hard to argue.

theory-and-practice-agree-here-in-my-experience-ly y'rs  - tim