No macros in Python

[Beni Cherniavsky]

On 2002-12-15, Mike Meyer wrote:

> Real macros change the language more fundamentally than that, by
> changing what is understood to be the syntax of the language. In
> python, keyword terminated by a ":" are part of the syntax. In the
> hypothetical macro version of python, that's changed.Macros can now
> fill that role. From one viewpoint, the macro adds syntax. From
> another, it doesn't.
>
Let me take a phylophical step aside.  In some way, the approaches of LISP
and Python are surprisingly similar.  One of the main arguments raised
against macros in Python is that code using them easily becomes unreadable
to one who doesn't know macros.  They feel that the uniform syntax of
python (a ``def`` is a function, etc.) protects them by ensuring some
basic common language for structuring the program.

Surprise: so do lispers!  They too felt that arbitrary use of source
transformation would render programs unreadable (see FORTH for example but
they also developed some consistency).  So they have a common platform for
structuring their programs -- S-Expressions.  And they adopted infomal
common idioms in macro design (`def*` define *s and have similar argument
order, `*let` have common argument structure, `with-*` are dynamic biding
wrappers, etc.) that help a lot (enough for editors to indent most unknown
macros correctly!)...  LISP macros are tamed by the underlying "Everything
is an S-expr" just as Python functions/classes/etc.  are tamed by
"Everything has the Python syntax".  The major difference is just the
choice of the common syntax.  Some would argue that LISPs have even more
fixed syntax than Python.

Some lispers (schemers) even made the macros forcibly hygienic.  The rest
didn't but -- behold -- not because they wanted all their macros to do
dirty things but bacause they learn to write more or less hygienic macros
without the cumbersome restrictions (using gensym and friends).

[Historical note: I distorted the cause-result relations: McCarty most
probably wasn't afraid of over-flexibility -- he put in as much as
possible; nevertheless sensible conventions arose that allow looking upon
code with well-desinged macro systems with less puzzlement than most
Python metaclasses (I'm not against metaclasses, on the contrary -- give
them some time and sensible results will arise too)]

[The existance of reader macros weakens my argument a bit but no lisper
will use them to e.g. allow unbalanced expressions.  Besides, after
reading everything is an S-expression anyway.]

> So you agree there are good uses for redefined operators. There are
> good uses for macros as well. You can also do Bad Things with
> functions, or even an if statement.

Like accessing the caller's locals() and writing into it.  That's no good
since Python has lexical scoping nowdays and the calling function can have
seveeral of them.  Which one should you access?  How will the parser know
that you are assigning to a variable in the caller and hence it should be
a local there?  The idea of ``locals(2)`` scares me more than Perl :-).
`eval` and `exec` have same shortcomings.

The point is that there are things missing in the language, and when there
are no macros, one starts writing ugly and mal-behaved code.  So a PEP is
created and a well-designed extension is added to the language on the next
release :-)  But until then, macros are a better fire-fighting tool than
the likes of `eval`.  Also if there were macros, some PEPs could get along
without meddling with the C code.

> Macros are a general-purpose mechanism. There are lots of things they
> let you do. Lazy evaluation, infinite lists, CLOS, short-circuit
> booleans, and so on. Some of these things are already in Python. Some
> are part of proposed future changes. Some may never be there at
> all. All the proposed changes are changing the interpreter and the
> syntax of the language to add a feature that could be implemented *in
> the language* if it had macros.
>
> Now, would you rather have one general-purpose mechanism that makes a
> lot of things possible, or a lot of special-purpose mechanisms?
>
[stealing the question from the person it was directed to ;-]

I'm of course in the "let me experiment" camp.  Look, did anyone here
(modulo the dev wizards) have to read a lot of code using metaclasses or
import hooks since they were added to the language?  How bad was it?  Of
course some was rather misleading, but some wasn't (and how much there was
at all?).  But there is no feature of Python that's only used in readable
ways.  The fact is, there are some very advanced abilities to morph code
already in Python and nobody suffers.  And some people are very happy
because they can e.g. experiment with only slightly suboptimal inteface
syntaxes today.

-- 
Beni Cherniavsky <cben at tx.technion.ac.il>