merits of Lisp vs Python

[Jon Harrop]

Wolfram Fenske wrote:
> Sure, you can shoot yourself in the foot with macros.  But you can do
> that in any language of any degree of expressiveness [3].  Come to
> think of it, the whole reason why we use high level languages is
> because of their expressiveness: We get stuff done faster and
> introduce less errors.  So "the more expressive, the better," right?
> But according to you, there's a point when a language gets "too
> expressive".  I don't see why.

Is Qi Lisp? There are several problems:

1. Language design is so hard that most people get it wrong (e.g. Python).
What do you do when you find that your proprietary syntax extension was a
bad idea?

2. There is a trade-off between being expressive and sacrificing syntax.
Lisp is very verbose because you must write the AST for your code by hand,
even if you aren't exploiting s-exprs, EVAL and macros.

3. One could argue that every syntax extension in Lisp makes a new language.
So Lisp is a heavily forked language. Is that a good?

Maybe if you're a lone coder and an expert in language design then you can
knock up a DSL and halve your working time using Lisp. In industry, you're
more likely to hire four people with half the expertise each and restrict
them to a well-known C-like syntax.

>> Now, if Lispers would say "Oh yes, macros give you great power, and
>> with great power comes great responsibility. Be careful." then, no
>> doubt, we'd take you guys more seriously.
> 
> Sure, it's a powerful tool but it's not *that* hard to use.

Look at all the threads in c.l.l where people have gotten confused by their
own macros. Look at the unboxing macro that Juho Snellman used to try to
get competitive performance for Lisp on my ray tracer benchmark:

(defmacro def ((name params &body body)
               (mname &rest mparams)
               (wname &rest wparams))
  `(progn
    (declaim (inline ,name ,wname))
    (defun ,name ,params
      (declare (type double-float , at params))
      , at body)
    (defmacro ,mname ,(mapcar #'car mparams)
      ,(loop with inner = (list name)
             with body = ``,',inner
             with all-names = nil
             for (form count) in (reverse mparams)
             for names = (loop repeat count collect (gensym))
             do
             (setf all-names (append all-names names))
             (setf body ``(multiple-value-bind ,',(reverse names)
                           ,,form ,,body))
             finally
             (setf (cdr inner) (reverse all-names))
             (return body)))
    (defun ,wname ,(mapcar #'car wparams)
      (,mname ,@(mapcar #'cadr wparams)))))

Note that this page of magic does something done by the compiler in most
other languages.

> Maybe you're afraid of it because that it's something that's unique to
> Lisp? 

It isn't unique to Lisp. OCaml has camlp4, for example. It is very rarely
used, perhaps because the OCaml community is wary of syntax extensions or
perhaps because it is even harder to use than Lisp's macros.

> But IMO the reason for that is not that they're too powerful.  IMO it
> has mostly to do with the fact that other languages' syntaxes make it
> too difficult to implement Lisp-style macros.

I agree. However, the fact that other languages ship with sophisticated
syntax can also be a good thing: it can make them more concise, it can make
them more expressive (e.g. pattern matching).

I get the impression that Lispers use macros when they could use HOFs.

>> But we don't hear that -- we hear Lispers going on and on about how
>> great it is that they can easily redefine every corner of the
>> language. Do you blame people for *believing them* and imagining
>> that reading Lisp code is like following some ghostly
>> will-o-the-wisp across a swamp, where nothing is what it seems and
>> the landscape is forever shifting?
> 
> Come on!  You're telling me people don't learn Lisp because they are
> afraid of it?  Python allows you to redefine built-in functions, as
> you said, Ruby allows you to attach new methods to live objects, but
> Lisp is simply going too far?

The biggest activation barrier to using Lisp for me is having to write ASTs
for everything by hand. Perhaps it would be a good idea to include an
editor that allowed newbies to use a more familiar syntax and have it
converted to s-exprs for them, so they can see the correspondence and learn
how to write s-exprs themselves?

>> Now, if you want to tell me that, despite all the talk, Lisp coders
>> don't actually create new syntax or mini-languages all that often,
>> that they just use macros as functions, then the question becomes:
>> why do you need macros then if you are just using them as functions?
>> Why not use functions?
> 
> Easy because macros are not functions.  Functions allow you abstract
> functionality, macros allow you abstract syntax.

While that is true, you've provided two examples to back up his point. In
another post you use the example of wrapping try .. finally which can be
done with a HOF instead of a macro. Next, you use the example of COND,
which can also be written as a HOF instead of as a macro:

> Look at the examples 
> above.  How would you implement conditional expressions as a function?
> Answer: You can't, it's syntax.

That is not true. You can implement conditional expressions without
extending the syntax:

# let rec cond x rules default = match rules with
    | [] -> default
    | (f, e) :: _ when f(x) -> e
    | _ :: t -> cond x t default;;
val cond : 'a -> (('a -> bool) * 'b) list -> 'b -> 'b = <fun>

For example:

# cond 2
    [( = ) 1, "one";
     ( = ) 2, "two";
     ( = ) 3, "three"]
    "neither one, two nor three";;
- : string = "two"

Of course, this is pointless in OCaml because the built-in pattern matcher
is more expressive, powerful and faster than COND:

# match 2 with
    | 1 -> "one"
    | 2 -> "two"
    | 3 -> "three"
    | _ -> "neither one, two nor three";;
- : string = "two"

-- 
Dr Jon D Harrop, Flying Frog Consultancy
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists/index.html?usenet