Python syntax in Lisp and Scheme

[Kaz Kylheku]

Matthias Blume <find at my.address.elsewhere> wrote in message news:<m1d6d9njkr.fsf at tti5.uchicago.edu>... 
> Well, no, not really.  You can define new syntactic forms in terms of
> old ones, and the evaluation rules end up being determined by those of
> the old ones.  Again, with HOFs you can always get the same
> effect -- at the expense of an extra lambda here and there in your
> source code.

A macro can control optimization: whether or not something is achieved
by that extra lambda, or by some open coding.

In the worst cases, the HOF solution would require the user to
completely obfuscate the code with explicitly-coded lambdas. The code
would be unmaintainable.

Secondly, it would be unoptimizeable. The result of evaluating a
lambda expression is an opaque function object. It can only be called.

Consider the task of embedding one programming language into another
in a seamless way. I want to be able to write utterances in one
programming language in the middle of another. At the same time, I
want seamless integration between them right down to the lexical
level. For example, the embedded language should be able to refer to
an outer variable defined in the host language.

HOF's are okay if the embedded language is just some simple construct
that controls the evaluation of coarse-grained chunks of the host
language. It's not too much of an inconvenience to turn a few
coarse-grained chunks into lambdas.

But what if the parameters to the macro are not at all chunks of the
source language but completely new syntax? What if that syntax
contains only microscopic utterances of the host language, such as the
mentions of the names of variables bound in surrounding host language?

You can't put a lambda around the big construct, because it's not even
written in the host language! So what do you do? You can use an escape
hatch to code all the individual little references as host-language
lambdas, and pepper these into the embedded language utterance. For
variables that are both read and written, you need a reader and writer
lambda. Now you have a tossed salad. And what's worse, you have poor
optimization. The compiler for the embedded language has to work with
these lambdas which it cannot crack open. It can't just spit out code
that is integrated into the host language compile, where references
can be resolved directly.

> > This can only be accomplished with functions if you're
> > willing to write a set of functions that defer evaluation, by, say
> > parsing input, massaging it appropriately, and then passing it to the
> > compiler. At that point, however, you've just written your own macro
> > system, and invoked Greenspun's 10th Law.
> 
> This is false.  Writing your own macro expander is not necessary for
> getting the effect.  The only thing that macros give you in this
> regard is the ability to hide the lambda-suspensions. 

That's like saying that a higher level language gives you the ability
to hide machine instructions. But there is no single unique
instruction sequence that corresponds to the higher level utterance.

Macros not only hide lambdas, but they hide the implementation choice
whether or not lambdas are used, and how! It may be possible to
compile the program in different ways, with different choices.

Moreover, there might be so many lambda closures involved that writing
them by hand may destroy the clarity of expression and maintainability
of the code.

> To some people
> this is more of a disadvantage than an advantage because, when not
> done in a very carefully controlled manner, it ends up obscuring the
> logic of the code.  (Yes, yes, yes, now someone will jump in an tell
> me that it can make code less obscure by "canning" certain common
> idioms.  True, but only when not overdone.)

Functions can obscure in the same ways as macros. You have no choice.
Large programs are written by delegating details elsewhere so that a
concise expression can be obtained.

You can no more readily understand some terse code that consists
mostly of calls to unfamiliar functions than you can understand some
terse code written in an embedded language build on unfamiliar macros.

All languages ultimately depend on macros, even those functional
languages that don't have user-defined macros. They still have a whole
bunch of syntax. You can't define a higher order function if you don't
have a compiler which recognizes the higher-order-function-defining
syntax, and that syntax is nothing more than a macro that is built
into the compiler which captures the idioms of programming with higher
order functions!

All higher level languages are based on syntax which captures idioms,
and this is nothing more than macro processing.