Python syntax in Lisp and Scheme

[Alex Martelli]

Pascal Costanza wrote:
   ...
>>>Well, this proves that Python has a language feature that is as
>>>dangerous as many people seem to think macros are.
   ...
>> Indeed, a chorus of "don't do that" is the typical comment each
>> and every time a newbie falls into that particular mis-use.  Currently,
   ...
>> large existing codebase) is under active consideration for the next
>> version of Python, expected (roughly -- no firm plans yet) in early 2005.
> 
> OK, I understand that the Python mindset is really _a lot_ different
> than the Lisp mindset in this regard.

As in, no lisper will ever admit that a currently existing feature is
considered a misfeature?-)


> Ah, you want something like final methods in Java, or better probably
> final implicitly as the default and means to make select methods
> non-final, right?

Not really, the issue I was discussing was specifically with importing.

Normally, an import statement "looks" for a module [a] among those
already loaded, [b] among the ones built-in to the runtime, [c] on
the filesystem (files in directories listed in sys.path).  "import hooks" 
can be used to let you get modules from other places yet (a database,
a server over the network, an encrypted version, ...).  The new architecture
I mentioned lets many import hooks coexist and cooperate, while the
old single-hook architecture made that MUCH more difficult, that's all.

"final implicitly as the default and means to make select methods
non-final" is roughly what C++ has -- the "means" being the "virtual"
attribute of methods.  Experience proves that's not what we want.
Rather, builtin (free, aka toplevel) _names_ should be locked down
just as today names of _attributes_ of builtin types are mostly
locked down (with specific, deliberate exceptions, yes).  But I think
I'm in a minority in wanting similar mechanisms for non-built-ins,
akin to the 'freeze' mechanism of Ruby (and I'm dismayed by reading
that experienced Rubystas say that freeze LOOKS like a cool idea
but in practice it's almost never useful -- they have the relevant
experience, I don't, so I have to respect their evaluation).


> What makes you think that macros have farther reaching effects in this
> regard than functions? If I call a method and pass it a function object,
> I also don't know what the method will do with it.

Of course not -- but it *cannot possibly* do what Gat's example of macros,
WITH-MAINTAINED-CONDITION, is _claimed_ to do... "reason" about the
condition it's meant to maintain (in his example a constraint on a variable
named temperature), about the code over which it is to be maintained
(three functions, or macros, that start, run, and stop the reactor), 
presumably infer from that code a model of how a reactor _works_, and
rewrite the control code accordingly to ensure the condition _is_ in fact
being maintained.  A callable passed as a parameter is _atomic_ -- you
call it zero or more times with arguments, and/or you store it somewhere
for later calling, *THAT'S IT*.  This is _trivially simple_ to document and
reason about, compared to something that has the potential to dissect
and alter the code it's passed to generate completely new one, most
particularly when there are also implicit models of the physical world being
inferred and reasoned about.  Given that I've seen nobody say, for days!, 
that Gat's example was idiotic, as I had first I thought it might be, and 
on the contrary I've seen many endorse it, I use it now as the simplest
way to show why macros are obviously claimed by their proponents to
be _scarily_ more powerful than functions.  (and if a few voices out of
the many from the macro-lovers camp should suddely appear to claim
that the example was in fact idiotic, while most others keep concurring
with it, that will scale down "their proponents" to "most of their
proponents", not a major difference after all).


> Overriding methods can also be problematic when they break contracts.

That typically only means an exception ends up being raised when
the method is used "inappropriately" - i.e. in ways depending on the
contract the override violates.  The only issue is ensuring that the
diagnostics of the error are clear and complete (and giving clear and
complete error diagnostics is often not trivial, but that is common to
just about _any_ classes of errors that programmers do make).

> (Are you also considering to add DBC to Python? I would expect that by
> now given your reply above.)

Several different implementations of DBC for Python are around, just
like several different architectures for interfaces (or, as I hope, 
Haskell-like typeclasses, a more powerful concept).   [Note that the
lack of macros stops nobody from playing around with concepts they
would like to see in Python: they just don't get to make new syntax
to go with them, and, thus, to fragment the language thereby:-)].

Guido has already declared that ONE concept of interfaces (or 
typeclasses, or protocols, etc) _will_ eventually get into Python -- but
_which one_, it's far too early to tell.  I would be surprised if whichever
version does make it into Python doesn't let you express contracts.
A contract violation will doubtlessly only mean a clear and early error
diagnostic, surely a good thing but not any real change in the
power of the language.


> Can you give an example for the presumably dangerous things macros
> supposedly can do that you have in mind?

I have given this repeatedly: they can (and in fact have) tempt programmers
using a language which offers macros (various versions of lisp) to, e.g.,
"embed a domain specific language" right into the general purpose language.
I.e., exactly the use which is claimed to be the ADVANTAGE of macros.  I
have seen computer scientists with modest grasp of integrated circuit design
embed half-baked hardware-description languages (_at least_ one different
incompatible such sublanguage per lab) right into the general-purpose
language, and tout it at conferences as the holy grail -- while competitors
were designing declarative languages intended specifically for the purpose
of describing hardware, with syntax and specifically limited semantics that
seemed to be designed in concert with the hardware guys who'd later be
USING the gd thing (and were NOT particularly interested in programming
except in as much it made designing hardware faster and cheaper).  The
effort of parsing those special-purpose language was of course trivial (even
at the time -- a quarter of a century ago -- yacc and flex WERE already
around...!), there was no language/metalanguage confusion, specialists in
the domain actually did a large part of the domain-specific language design
(without needing macro smarts for the purpose) and ended up eating our
lunch (well, except that I jumped ship before then...;-).

Without macros, when you see you want to design a special-purpose
language you are motivated to put it OUTSIDE your primary language,
and design it WITH its intended users, FOR its intended purposes, which
may well have nothing at all to do with programming.  You parse it with a
parser (trivial these days, trivial a quarter of a century ago), and off you 
go.  With macros, you're encouraged to do all the wrong things -- or, to
be more precise, encouraged to do just the things I saw causing the
many costly failures (one or more per lab, thanks to the divergence:-)
back in that my early formative experience.

I have no problem with macros _in a special-purpose language_ where
they won't tempt you to embed what you _should_ be "out-bedding",
so to speak -- if the problem of giving clear diagnostics of errors can be
mastered, denoting that some functions are to be called at compile
time to produce code in the SPL has no conceptual, nor, I think,
particular "sociological" problem.  It's only an issue of weighing their
costs and usefulness -- does the SPL embody other ways to remove
duplication and encourage refactoring thereby, are there overlap
among various such ways, etc, etc.  E.g., a purely declarative SPL,
with the purpose of describing some intricate structure, may have no
'functions' and thus no other real way to remove duplication than
macros (well, it depends on whether there may be other domain
specific abstractions that would be more useful than mere textual
expansions, of course -- e.g. inheritance/specialization, composition
of parts, and the like -- but they need not be optimal to capture some
inevitable "quasi-accidental duplications of SPL code" where a macro
might well be).


Alex