Python syntax in Lisp and Scheme

[Doug Tolton]

On Wed, 08 Oct 2003 14:22:43 GMT, Alex Martelli <aleax at aleax.it>
wrote:

>Doug Tolton wrote:
>   ...
>> Alex, this is pure un-mitigated non-sense.
>
>Why, thanks!  Nice to see that I'm getting on the nerves of _some_
>people, too, not just having them get on mine.

Yes, this discussion is frustrating.  It's deeply frustrating to hear
someone without extensive experience with Macros arguing why they are
so destructive.  Particularly to hear the claim the Macros make it
impossible to have large teams work on a project, and at the same time
supporting features in python that make it far more difficult to share
code between people.  (ie, white space delimiting, the ability to
rebind interenals, Metaclasses).  Personally I'm not against any of
these features, however they all suffer from serious potential
drawbacks.
>
>> Python's Metaclasses are
>> far more dangerous than Macro's.  Metaclasses allow you to globally
>> change the underlying semantics of a program.
>
>Nope: a metaclass only affects (part of the semantics of) the
>classes that instantiate it.  No "globally" about it.  When I
>write a class I can explicity control what metaclass it uses,
>or inherit it.  E.g., by writing 'class My(object):', with no
>explicit metaclass, I ensure type(my)==type(object).  The type
>of the built-in named 'object' is the built-in named 'type'
>(which most custom metaclasses subclass), which is also the
>type of most other built-in types (numbers, strings, list,
>tuple, dict, functions, methods, module, file, ...).  I.e.,
>your assertion is pure un-mitigated FUD.
>
Please explain to me how according to your logic, a semantic change to
the language is good, but a syntactic change is bad.  Your logic is
extremely confusing to me.  On the one hand you think Macro's are bad
because I can define a construct such as 
(do-something-useful-here
	arg1
	arg2
	arg3)
which operates according to all the regular semantic rules.  In fact
there is even an explicit construct that will show me *exactly* what
this code is doing.  Yet you apparently think using Metaclasses to
change the underlying semantics is somehow ok, because you can check
to see if it's built-in?  So are you saying that using only built-in
constructs are good to use?  If someone else gives you a class to use
which uses Metaclasses to change how it operates for some reason or
another, are you ok with that?  What if they need to re-bind some of
the builtins to do something? Because you can't prevent that in python
either.  In fact any piece of python code that runs on your system
could do that, yet you are ok with that?

>> Macros only allow you
>> to locally change the Syntax.
>
>"Locally"?  Not globally?  Care to explain?  Each 'invocation'
>(expansion) of a macro must occur in a particular locus, sure,
>but isn't the _point_ of (e.g.) defining a 'loop-my-way' macro
>that every occurrence of 'loop-my-way' is such an expansion?
yes, but the point is that if I just wrote 'loop-my-way', it doesn't
change existant code in unexpected ways.  It only affects new code
that is written using 'loop-my-way'.  Whereas re-binding the buildins
*will* change existant code.
>
>As for that mention of "the Syntax" AS IF somehow contrasted
>with the "underlying semantics" just before, don't even bother
>to try explaining: one example of macros' wonders offered by a 
>particularly vocal and emotional advocate was a macro
>'with-condition-maintained' that was somehow supposed to make
>whatever alterations might be needed in the control program of
>a reactor in order to regulate temperature -- and it was
>passed that code as three calls to functions (or expansions
>of macros) NOT defined inside it, so how it could possibly
>work "only...locally", when to do anything at all it MUST
>necessarily find, "expand", and alter the local instantiations
>of those functions (or macros)...?!
>
I'm not going to attempt to explain with-condition-maintained, as you
are clearly ignoring the general idea in favor of nitpicking
non-essential details of a hypothetical construct.  When you descend
to the statements you have made about that construct, it's no longer
worth continuing discussion of it.

That's my exact problem though, your statements continually brand you
as ignorant of what macros are and how they operate on a fundamental
level, yet for some reason you feel qualified to extol their evils as
if you actually have significant experience with them.

>If that's an example of "only allow you to locally change
>the syntax", what would be an example of *EVEN DEEPER
>AND MORE PERVASIVE* changes ?!
>
No idea what you are talking about here.

>> Your comparison is spurious at best.
>
>What "my comparison" are you blabbering about?  My text that
>you quoted, and called "pure un-mitigated non-sense", had no
>"comparisons", neither my own nor others'.  I see YOU attempting
>to draw some "comparison" (eminently spurious, to be sure) 
>between metaclasses and macros...
The only reason you think it's spurious is because of your
fundamentally flawed conception of Macros.
>
>
>> Your argument simply shows a serious mis-understanding of Macros.
>> Macros as has been stated to you *many* times are similar to
>> functions.  They allow a certain type of abstraction to remove
>> extraneous code.
>
>Yeah, right.  Kindly explain that 'with-condition-maintained'
>example and the exhalted claims made and implied for it, then.
I thought you didn't want it explained to you?  If you are serious
about wanting to know what the *point* of the code snippet was, then
I'll explain it.  If on the other hand you are going to make some
ridiculous argument about how with-condition-maintained isn't in fact
hooked to any control room circuitry, then forget it.
>
>
>> Based on your example you should be fully campaigning against
>> Metaclasses, FP constructs in python and Functions as first class
>> objects.  All of these things add complexity to a given program,
>
>"FUD" and "nonsense" (with or without a hyphen) would be serious
>understatements in an attempt to characterize *THIS*.  *HOW* do
>"functions as first class objects" perform this devilish task of
>"adding complexity to a given program", for example?!  The extra
>complexity would be in rules trying to FORBID normal usage of an
>object (passing as argument, returning as value, appending to a
>list, ...) based on the object's type.  There is obviously no
>complexity in saying "_WHATEVER_ x happens to stand for, you
>can correctly call somelist.append(x)" [passing x as argument to
>a method of the somelist which appends x to the list], for example.
>The added complexity would come if you had to qualify this with
>"UNLESS ..." for whatever value of ``...''.
>
Hmm...what if using a class with Metaclasses behaves in a totally
non-standard way?  What if a function re-binds the builtins?  What if
they over use FP constucts and nest 50 maps and filters?  Are you ok
with all of these things?  They are certainly more confusing than
Macros.  To make the statement that *any* technology can't be abused
is foolish.  To make that claim implies there is no correct usage,
only usage.  In other words if there is no correct way to use
Metaclasses or re-bind builtins then any way that someone sees fit to
do it *is* the right way.  We all know that is a ridiculous claim.
Macros are like any other sufficiently powerful technology.  If they
aren't used right, they will complicate a program not simplify it.

I believe the crux of our difference is that you don't want to give
expressive power because you believe it will be misused.  I on the
other hand want to give expressive power because I believe it could be
used correctly most of the time.  For the times when it's not, well
that's why I have debugging skills.  Sadly not eveyone uses looping
the way I would, but using my brain I can figure out what they are
doing.
>> however they also reduce the total number of lines.  Reducing program
>> length is to date the only effective method I have seen of reducing
>> complexity.
>
>For some (handwaving-defined) "appropriate" approach to measuring
>"length" (and number of lines is most definitely not it), it is ONE

Both from my experience and Fred Brooks it's the only actual way I've
seen of measuring the time it will take to write a program.

>important way.  But you're missing another crucial one, which is
>the count of interactions, actual and potential, between "parts"
>of the program -- the key reason why global effects do not in fact
>effectively reduce complexity, but rather bid fair to increase it,
>even though they might often "reduce the total [[length]]", is
>exactly this. 

I understand this point very well.  That's why I believe in building
layered software, and using good high order constructs to acheive
this.  As I've said before, your statements reveal your fundamental
mis-understanding of the way Macro's work.  To support Metaclasses,
classes, functions, first order functions etc as tools to support this
concept while at the same time reviling macros is simply showing an
un-educated bias about Macros.  I wouldn't be suprised to hear you
respond with some argument about how you've read the writings by
people who have used Macros (as you've done in the past), but I
believe you do not have sufficient understanding to make the claims
you are making.  If you really understood macros, I don't believe you
would be making such statements.

> E.g., if large parts of my program needed all kinds
>of comparisons between strings (including comparison-related
>functionality such as hashing) to be case-insensitive, it might
>make my program 'shorter' if I could set case insensitivity as
>the global default -- but it might easily mess up totally unrelated
>and otherwise stable modules that rely on the usual case sensitive
>operations, causing weird, hard-to-trace malfunctionings.  I've
>mentioned my youthful APL experiences: with its quad-IO to globally
>set index origin for arrays, and its quad-I forget what to globally
>set comparison tolerance in all comparisons between floating point
>numbers, APL was a prime example of this (among other things,
>reusability-destroying) global-effects risk.  Sure, it was cool
>and made my program shorter to be able to check if "a < b" and
>have this IMPLICITLY mean "to within N significant digits" (or
>whatever) -- but it regularly broke other otherwise-stable modules
>and thus destroyed reuse.  Not to mention the mind-boggling effects
>when a<b, a>b and a=b can ALL be 'true' at once thanks to the
>"to within N significant digits" IMPLICIT proviso...
>

Well that was a long winded digression into something that is
completely un-related to Macros.  Seems like a good argument why
re-binding the buildins is bad though
.
>Complexity is not just program length, and reducing program length
>not the only important thing in reducing complexity.  Removing
>*repetition* (boilerplate), sure, that's nice -- and if there was
>a way to constrain macros to ONLY do that (as opposed to ending up
>with examples such as 'with-condition-maintained', see above) I
>would be very interested in seeing it.  I doubt there is one, though.
>
I agree that reducing complexity is the goal.  I disagree that you can
*ever* guarantee a high order construct is always used correctly
though.
>
>> If you truly believe what you are saying, you really should be
>> programming in Java.  Everything is explicit, and most if not all of
>
>Hmmm, one wonders -- are you a liar, or so totally ignorant of what
>you're talking about that you don't even KNOW that one of Java's
>most "cherished" features is that the "self." is just about ALWAYS
>implicit...?  Anyway, in my text which you quoted and characterized
>as "pure un-mitigated non-sense" I was speaking of UNIFORMITY as
>a plus -- and Java's use of { } for example ensures NON-uniformity
>on a lexical plane, since everybody has different ideas about where
>braces should go:-).
>
Where braces should go is a trivial issues.  However if braces is an
issue that seriously concerns you then I can see why macros are giving
you a heart attack.

>But I've NEVER argued in favour of boilerplate, of repetitiousness.
>I think that the occasional error that you can catch by forcing
>redundancy is generally outweighed by all the errors that just
>would not be there if the language let me state things "once, and
>only once".  So, for example, when I write
>    x = 23
>I most definitely don't WANT to have to redundantly state that,
>by the way, there is a variable x, and, whaddyaknow, x refers
>to an integer.  As to whether it makes more sense to later let
>the same name x in the same scope refer to OTHER objects (of
>the same type; or, of any type) -- I still don't know; maybe
>a single-assignment kind of functional language would in fact be
>preferable, or maybe Python's relaxed attitude about re-bindings
>is best, or maybe something in-between, allowing re-bindings but
>only within a single type's items (for "re-bindings" you may
>choose to read "assignments" if you wish, I'm not trying to
>reopen THAT particular lexical flamewar for further debate;-).
>
>So far, I'm pretty happy with Python's permissive approach to
>mutation and re-binding, but I notice I don't mind (differently
>from many others) the inability to re-bind SOME references
>(e.g., items of tuples, or lexically-outer names) -- and in
>Haskell or ML I don't recall ever feeling confined by the
>inability to have the same name refer to different values at
>successive times (in the same scope).  [I _do_ recall some
>unease at being unable to mutate "large" data structures, as
>opposed to rebinding simple names, so it's not as if I can
>claim any natural affinity for the functional [immutable-data]
>approach to programming -- I just wonder if perhaps the current
>widespread _emphasis_ on rebinding and mutation may not be a
>TAD overdone -- but, enough for this aside].
>
>I do, of course, truly believe in what I'm saying -- what
>WOULD have stopped me from taking up any of a zillion different
>languages, instead of Python, when I started studying it
>about four years ago?  Indeed, my opportunities for making
>money, and the audience for my books, would be vaster if I
>had stuck with what I was mainly using at work then (mostly C++,
>some Java, VB, Perl), my academic respectability higher if I
>had stuck with Haskell or some ML.  But while I don't mind
>money, nor fans, I care most about other values -- and the
>amount to which "Python fits my brain" and makes me most
>comfortable and productive meets and exceeds all claims I had
>heard to this effect, PLUS, I have experiential proof (enough
>to convince me personally, if nobody else:-) that it's just
>as comfortable and productive for many others, from programming
>newbies to highly experienced professionals.  Sure, Java would
>let me program my cellphone (which currently doesn't support
>Python) -- oh well, I'll have to eschew that crucial pursuit
>for a while longer now...
>
>
The ironic thing is that I'm not bashing Python.  I really like
python.  It's a great language.  I think we both use Python and Lisp
(for me) for the same reasons.  If I wanted a higher paying job I'd be
using Java.  I have aspirations to write books as well, I agree that
Python and Lisp aren't the biggest markets, and yet I use them because
they fit my brain also.

What I am in point of fact upset by is your constant barrage against
Macros.  I feel that your stance is based on ignorance and
mis-information.  You certainly don't have significant first hand
exposure to Lisp style Macros or you wouldn't be making statements
that were so obviously incorrect.  Why don't you seriously try to
learn them?  If you don't care to, why argue about them so much?  I
haven't seen anyone bring up (I could've missed it) putting Macro's
into Python again.  I personally don't think Macros would work very
well in Python, at least not as well as they do in Lisp.  So
understanding that I'm not pushing for Macro's in python, why are you
so vehement against them?  Are you on campaign to get Macros out of
Lisp?

Doug
Doug Tolton
(format t "~a@~a~a.~a" "dtolton" "ya" "hoo" "com")