do...until wisdom needed...

[Andrew Dalke]

Me:
>> Perhaps, but my feeling is that the level of training you need
>> for that is greater than most of my clients - chemists and
>> biologists - want to go through.  Note that since I've never
>> used a language (other than Tcl?) with these sorts of macros
>> I don't have specific knowledge of the pros and cons.

Douglas Alan replied:
>Procedural macros are best left to the experienced professional
>programmer to implement.  The more casual programmer can significantly
>benefit from the work of the expert, though.

Hmmm. I am an experienced professional programmer, at least
for my field.  And I don't want to touch procedural macros.  From
what I've heard from others in private mail, too often non-
"experienced professional programmers" end up making their own
dialect, and the result is a Babel of code.  As one person
commented to me:

] If you give people extensible syntax, and Andrew and I adopt
] different dialects of python, then instead of getting a bug
] fix, I will get, '`I am sorry, but I don't have time to learn
] your macro system.' or even worse, `I haven't used regular
] python for so long, that I forget how it works'.  This is
] precisely what happened to me [..when I..] decided writing my
] own lisp macros to do physics in was a good idea.  It was a
] terrible, terrible, horrible mistake.  I doomed myself to
] doing ALL the physics in the world.

My main expertise is developing infrastructures that allow
widely different tools to interoperate.  As such, I aim to
reduce the amount of variety so people can work on getting
their research done, not figuring out how to use the tools.
Nearly everything I hear suggest that procedural macros
don't help reduce complexity.

I still maintain that having the Python source code available
gives at least comparable advantages to procedural macros.
If someone wants to implement new behaviour, it can be done
without all that much work.  The big advantage is that all of
these changes don't get propagated willy-nilly.  Either you
take the hit of distributing a new Python-derived language
(as with Alice) or it's good enough that Guido accepts it
into the core, in which case it isn't a dialect.  In either
case, it makes you think strongly about the worth of the new
language extension.

Yes, doing this required using a non-Python language for the
implementation, but as I've said I don't believe in restricting
myself to one language.  Yes, it reduces the ability of just
anyone to modify the language, but you are already saying the
only ones qualified to change a language are "experienced
professional programmer"s, and they should have no problem with C.
Yes, it calls for more work to implement, but I'm not convinced
it should be easy.

Other ways to get around the lack of procedural macros were
mentioned (besides changing the language core):
   - calling a library function to parse an alternate language
   - importing the new code via a modified import hook
       (this could be made easier with the work on a library
        to build ASTs for Python)
   - writing a C extension which uses the metaobject interface,
        as with the Persistence type in Zope.  (This can only
        change the object behaviour.)

So who all is affected by the lack of procedural macros in
a way that cannot be supported by one of these mechanisms?  I
suspect it's a small audience.  Not to say it wouldn't be
an influential audience, just that it would be small.

> The same thing is often true for classes

That's true.  My clients don't have much of a problem using
classes/instances, but the ones without much CS training are
somewhat hesitant writing their own classes.  But the
complexity of classes seems to be much less than procedural
macros, since people who aren't professional programmers can
still learn to write them well enough.

Me:
>> I can also point out the almost complete nonexistence of a major
>> package in computational chemistry or biology written in a language
>> supporting hygenic/procedural macros as an indicator that
>> *something* prevents their wider use in my field.

>There aren't any widely used languages that have procedural macros
>other than Lisp and its variants.

That's almost my point, but the other way around.  If procedural
macros are the cat's meow then you'ld think that someone else
beside the Lisp'ers would implement it.  So either a) new ideas
take a long time to become mainstream, so wait another decade,
b) the rest of the world is too ignorant/ conservative/
unambitious/ etc. c) there is a problem with the usefulness
of procedural macros in mainstream development or d) it cannot
be done in any language other than Lisp.

My perception of things weighs things more towards c).  It seems
that you weigh b) stronger than I do, with some complaint about a).
I still can't figure out how address the d) part without basically
quoting everything, in which case it's identical to having a
function that parses a string and exec's the result.

>And Lisp only really ever caught on in AI circles
>and for teaching programming at finer institutions of higher
>education.  The reasons for that are complex and I think don't say
>much about the pros and cons of procedural macros, or even of Lisp in
>general.

"finer institutions of higher education"?  Seems a rather biased
phrase.  Bet my alma mater's football team could beat yours! :)
And its music department, and its film school, and its synchronized
swimming team, and its hospitality management program, and its
school of library science ... oh yeah, and its magnet lab (sorry,
that's an eight year old jibe :)

                    Andrew "Go 'Noles!" Dalke
                    dalke at acm.org