Why is tcl broken?

[Pierre R. Mai]

Paul Duffin <pduffin at mailserver.hursley.ibm.com> writes:

> If they are statically scoped how does the following work.
> 
> Look at the following code.
> 
> 	(let ((a 2))
> 	  (let ((b 3))
> 	    (+ a b)))
> 
> When you expand the macro let you get something like.
> 
> 	((lambda (a)
> 	   ((lambda (b) 
> 	      (+ a b)) 3)) 2)
> 
> Which is equivalent (apart from side effects) of.
> 
> 	(defun inner (b)
> 	  (+ a b))
> 
> 	(defun outer (a)
> 	  (inner 3))
> 
> 	(outer 2)

They are _NOT_ equivalent, because the reference to a in inner is now
completely free, which it was not in the lambda formulation.  When you
change the nesting of functions/forms like this, you of course change
their meaning (this is basic Lambda-calculus).

> So what is the process which associates the use of "a" inside inner
> with the formal argument of outer. I 

In lexical scoping there is no such mechanism, since a variable has to 
be lexically apparent, for it to be accessible.  Since a is not
lexically apparent in inner, it is not accessible.  That is one of the 
important features of lexical scoping, since it ensures that I only
have to look at the lexically enclosing environment to understand
which variables are accessed.

Common Lisp allows dynamic scoping for certain, specially declared,
variables.  In CL you _could_ write the following as a somewhat
equivalent form of your TCL code below, but no Lisp programmer would
do it that way:

(defun inner (b)
  ;; Note that a is a free reference in here, so we rely on there
  ;; being a dynamic binding in effect for a when we are called.
  ;; This is clearly insane ;)
  (+ a b))

(defun outer (a)
  (declare (special a))
  (inner 3))

(outer 2)

=> 5

If you really wanted to do something this way in CL, then a would be
considered a global parameter to the behaviour of inner, and would be
declared prominently as such.  If that isn't the case (as in your
example, where arguably a is just a parameter of inner that you don't
care to pass explicitly), you would simply use closures/nested
functions as appropriate, or indeed pass the argument around, if you
want inner to be self-sufficient.  I.e. either do

a)

(defun outer (a)
  (flet ((inner (b) (+ a b)))
    (inner 3)))

b) 

(defun inner (a b)
  (+ a b))

(defun outer (a)
  (inner a 3))

or 

c)

(defvar *inner-frobnication-parameter* 0
  "This global parameter allows you to fine tune the frobnicator of
inner in a global fashion.  Please use with appropriate care, since
too high value here might result in the sudden destruction of the
whole of creation, which some would consider to be enormous lossage.")

(defun inner (b)
  "Frobnicates b via a general purpose frobnication-algorithm.  See
also *inner-frobnication-parameter*, which gives the total bias for
the algorithm.  See Knuth, Vol 1, P. 169 for a specification of the
algorithm and the parameters involved."
  (+ *inner-frobnication-parameter* b))

(defun outer (a)
  "Frobnicates 3 via inner, with a as the setting for the
*inner-frobnication-parameter*."
  (let ((*inner-frobnication-parameter* a))
    (inner 3)))

> The Tcl equivalent is
> 
> 	proc inner {b} {
> 	    upvar 1 a a;
> 	    expr {$a + $b}
> 	}
> 
> 	proc outer {a} {
> 	    inner 3
> 	}
> 
> 	outer 2

Even though I'd shoot every programmer that wrote the Lisp code I
first wrote above, it still seems to me to be a much better construct
than the corresponding upvar in Tcl:

a)  With Tcl it seems to me that the caller has no way of knowing
    which of the procedures he calls might simply start poking at his
    variables.  Worse:  Since upvar can be done for any n levels, he
    will have to inspect _all_ of the procedures ever called while he
    is active, to ensure that they are not using any of his internal
    variables.

    This to me seems clearly insane:  If I were maintaining the Tcl
    code above, and at some point decided that a was a misnomer and
    should be called first-of-two (or whatever else), I'd have to look 
    at all procedures ever called to ensure none was relying on my
    having a variable called a.  In Lisp the special declaration would 
    at least warn me about the fact that someone somewhere was relying 
    on me dynamically binding a.

b) The fact that upvar has to specify at which level the
   corresponding variable is to be found, makes apparent it's hackish
   nature.  Inner shouldn't have to know who exactly will provide it
   with the value for a.  In Tcl I can't call inner via a mediating
   function, like this:

   proc master {a} {
     middle 3
   }

   proc middle {c} {
     inner c
   }

   For this to work, I'd either have to modify inner (in which case
   the old call-chain via outer would cease to work), or I'd have to
   modify middle, to ensure middle passes along a.  Now imagine master 
   either calling inner directly or via middle (depending on some
   condition), and you get into serious trouble understanding and
   maintaining the various relationships and variable bindings
   involved.  This seems to me to take the problems of dynamic binding 
   to new levels (i.e. it somehow reminds me of assembler code poking 
   around in a caller's stack frame, to avoid copying values into new
   stack frames).

> As I have said before it is simply an explicit form of the Lisp
> mechanism which allows the let macro to be implemented using
> lambda. It is very rare that N is more than 1.

To repeat it: dynamic scoping is in no way necessary or even helpful
for implementing let via lambda.  LET is directly expressible via
LAMBDA in the lambda-calculus, which is lexically scoped.  The same
goes for Scheme, which doesn't have dynamic scoping either.  What you
are doing is performing invalid transformations on lambda-calculus
expressions, going on to claim that since you need dynamic scoping to
make those transformations work, that dynamic scoping is needed to
allow LET to be expressed in LAMBDA.  Had you any knowledge of Lambda
calculus, you wouldn't make these ridiculous claims.  BTW: The invalid
transformations you perform are common-place in Tcl, because Tcl (like
C) doesn't allow nested functions.

> > You are very mistaken.  Common Lisp macros are definitively not
> > another language.  They manipulate S-expressions, which are what Lisp
> 
> Lisp macros manipulate S-expressions but the macro definitions themselves
> are interpreted differently to Lisp expressions.

The macro-definitions are interpreted exactly in the same way as any
other Lisp expression, i.e. in

(defmacro blabla (a b)
  (my-expression))

(my-expression) is interpreted in exactly the same way as
(my-expression) would in any other context (with appropriate bindings
in place of course).

> A simple macro version of setq would convert from
> 	(setq symbol '(list))
> to
> 	(set 'symbol '(list))

It wouldn't do this, since set can only access "global variables",
whereas setq can access local variables.  Take note please that set
is deprecated, and that setq is not a macro, but a special form.  If
this doesn't mean anything to you, then please refer to the ANSI
Common Lisp standard (also available in a hypertext version, at
http://www.harlequin.com/books/HyperSpec), which would benefit this
discussion greatly.

> A macro setq is not interpreted the same way as set is because if it
> was an error would occur when the Lisp interpreter tried to get the
> value of the variable symbol before symbol was created.

This doesn't make any sense.  Please get your vocabulary right.  The
symbols in question are created at read-time, so there is no way that
either of the above forms could be executed in an environment where
the symbol in question didn't exist, unless you did some very evil
hacking with the package system, the consequences of which I wouldn't
claim to be well-defined.

Now that I've re-read your sentence for the xth time, it seems to me
that you are claiming that (setq symbol 5) is interpreted differently
than (set symbol 5), which is of course right.  That was the whole
point of defining a macro for setq, that transformed (setq symbol 5)
into (set 'symbol 5) and not into (set symbol 5).

But this is not inconsistent in my book, because that is exactly what
you asked for: You _wanted_ every (setq a b) form to be interpreted
as if (set 'a b) had been written.  So after macro-expansion, the
evaluation of (set 'a b) is totally consistent with the evaluation of
any other set form.

If you find this inconsistency troublesome, than you have to condemn
every kind of macro system, and Tcl, too, which allows the user to
change the behaviour of commands.

> I don't underestimate the power of Lisp macros I am just saying that I
> find the way they warp the otherwise simple Lisp syntax / semantics.
> 
> Lisp does not really need macros, they are really just a way of
> creating short cuts.

Firstly this sounds like the old argument against HLLs, i.e. we don't
need high-level languages, they are really just a way of creating
short cuts, which though true, totally misses the point of HLLs.

Secondly, since special forms (of which setq as well as quote and
lambda are prime examples) also warp the otherwise simple evaluation
rules of Lisp, it can be argued that Lisp effectively needs at least a
certain amount of warping, to be usable at all.  Even the pure lambda
calculus is warped in your eyes then.  Funny how all the warpiness of
Tcl doesn't grate on your nerves...

> Tcl does not need a macro system because it has a simple consistent
> syntax / semantics and all commands are treated equally and it is
> possible to change the behaviour of existing commands.

This is clearly no argument at all, because all the purported features
of Tcl you mention are clearly orthogonal to the existence of a macro
system.

> I would say that Tcl (without macro system) can do anything that Lisp
> (with macro system) can do.

Assembler can do anything either Lisp or Tcl can do.  This is no
argument against either Lisp or Tcl.  At the point where programming
language discussions turn to this Turing-machine-equivalence argument, 
it is clear that the discussion is fruitless.

end-of-line.

Regs, Pierre.

PS:  Should any further discussion seems sensical (it doesn't to me),
please heed the F'up to c.l.l, since neither Python nor Scheme are at
all relevant to this.

-- 
Pierre Mai <pmai at acm.org>         PGP and GPG keys at your nearest Keyserver
  "One smaller motivation which, in part, stems from altruism is Microsoft-
   bashing." [Microsoft memo, see http://www.opensource.org/halloween1.html]