Loops and stuff (was Re: Python and Boehm-Demers GC, I have code.)

[Tim Peters]

[Michael Hudson]
> ...
> Python is not a functional programming language.

NOW they tell me <wink>.

> As I understand it, you can't really have convenient closures because it
> is not lexically scoped (I can give you inconvenient closures...) and
> there were lengthy discussions about that some months back. I didn't
> follow it too closely because I didn't know what lexically scoped meant
> then, but the upshot was (I think) that lexical scoping isn't really
> the "Python way".

It wasn't, but it may be.  Python's 3-level scoping system (which Guido
routinely calls a 2-level system these days -- I guess because he thinks 2
sounds less contrived than 3 <wink>) was a thoroughly deliberate decision.
Today he'll tell you that he may have overreacted against the rampant abuse
of deep procedure nesting common in large Pascal programs of that long-ago
time.  I tangled with him on this in email at the time, but having also
suffered the Pascal abuses, reluctantly agreed to stop harassing him about
it and just see how the 3-level scheme worked out in practice.

I think it worked out great!  I was surprised.  But there were two miserable
problems:

1) Despite its simplicity, the implementation of name resolution was much
slower than in most languages that support arbitrarily deep lexical nesting.
By far the worst of this got repaired by assigning locals to fixed slots
(and bytecodehacks wouldn't be nearly as much fun without that to abuse
<wink>).

2) Despite that scoping is 3-level, the *syntax* allows nesting functions &
classes to any level.  This is an eternal source of confusion for people
coming from other languages, and can take a long time to stop tripping over.
So why did the syntax allow it?  Because it would have complicated the
grammar to stop it!  Guido may deny that today, but it's the truth <wink>.

Times are different now.  The only people *likely* to abuse lexical nesting
these days are Scheme/Lisp heads, but Python's lack of a macro system will
drive them away anyway <wink>.

As far as the implementation goes, arbitrarily deep lexical closures with
indefinite extent are easy in Python -- it's a small & simple patch
(frames-- activation records --are heap-allocated anyway; all that's really
missing is a "static link" to crawl up the lexical nest).  Guido even
implemented (but never released) it a few years ago.  An unexpected
technical glitch killed it:  it turned out that the natural implementation
created cyclic implementation structures that refcounting isn't strong
enough to clean up.  Presumably Python2 could repair that by hook or by
crook.

So Schemish lexical scoping may yet become The Python Way!  I wouldn't count
on it, though; and maintaining state via class instances will always be more
Pythonic.

[prematurely wise comments elided]

> not-OO-functional-or-fast-but-still-great-ly y'rs
> it's-not-OO-or-functional-it's-Python-ly y'rs

it's-guido's-way-or-the-highway-ly y'rs  - tim