What's better about Ruby than Python?

[Andrew Dalke]

Jacek Generowicz:
> The alternative is to understand (and subsequently recognize) the
> chunks of source code implementing a given patten for which no
> abstraction was provided (often implemented slightly differently in
> different parts of the code, sometimes with bugs), each time that it
> occurs.

Agreed and understood.  My understanding of macros were that
they provide flexibility beyond what classes could do.  However,
at present, the only example I've seen for when to use a macro
came from a method cache implementation that I could implement
in Python using the normal class behaviour, so I don't have a
good idea of when macros would be appropriate *for* *Python*.

When this topic has come up before, others mentioned how
macros would theoretically be able to, say, modify list.sort
to return the sorted list after it has been modified in-place.

Given the not infrequent request for the feature, I know that
if it was allowed, then some of my clients would have done
that, making it harder for me to know if what I'm looking at
is core Python behaviour or modified.

What you say is true, but most of the code I look at is
based on fundamental Python types, from which I can
be assured of fixed behaviour, or classes and functions,
where I can be assured that they are free to do their
own thing.  The patterns of behaviour are fixed and
the opportunities for change well defined.

Macros, as I understand it, blurs those lines.

> > The inference is that programming language abstractions should not
> > be more attractive than sex.
>
> Why ever not?  Don't you want to put the joy back into programming :-)

Mmmm, I've reached the point in my life where the importance
of social interactions and community building is starting to outweigh
my interests in programming all the time.

Plus, programming in Python is pretty joyful.  It fits my mind
quite nicely.

> I don't believe you can reasonably draw a rigid and well-defined
> boundary between functions, modules and objects on one side, and
> macros on the other. They all offer means of abstraction. All are open
> to abuse. All can be put to good use.

I never said macros couldn't be put to good use.

Let me repeat a statement I made in another post on this topic.

  I will grant that Lisp or Scheme is the end-all and be-all of
  languages.  Where's the language between those and Python?

Is it possible to have a language which is more flexible than
Python but which doesn't encourage the various dialectization
historically evident in the Lisp/Scheme community?

Could most macros also be written without macros, using
classes or lambdas?  How often are the benefits of macros
that much greater than classes&functions to merit their
inclusion.

Does it take skill to know when to use one over the other?
Do people use macros too often?  When do they hinder
misunderstanding?  Are they more prone to misuse than
classes&functions?

You say that macros can be put to good use, *and I
believe you*.  I say that their specific advantages over
classes is rare enough that having an extra mechanism
for implementing behaviours does not warrant their
inclusion into Python - a language designed with an
emphasis on readability and usability over the extreme
flexibility emphasis from Lisp - given the statements
made by people here with strong Lisp backgrounds
on how using those seductive macros made it harder
to share code with outsiders.

> In all four cases, I'd rather have the opportunity to create
> abstractions, rather than not.

The complaint about macros has been their tendency to
increase a single person's abilities at the cost of overall
loss in group understanding.  I've heard references to
projects where that didn't occur, but am not swayed
by it because those seem staffed by people with
extraordinarily good programming skills almost never
found amoung the chemists and biologists I work with.

> I find your suggestion that macros are in some way more "domain
> specific" than modules, or objects or functions, bogus.

Sorry?  I thought one of the main points of macros is that
they allow additional flexibility to customize the language as
appropriate for a given domain.  Now you say that that's
not the case?

> You are confusing the issues of
>
> - extensibility,
> - standard non conformance,
> - not starting from a common base,
> - languages defined my their (single) implementation.

Indeed, and to some extent deliberately.  The point I'm
trying to make is that different, very smart people like
"Lisp", but insist on variations.  There is clisp and elisp
and scheme and guile and ... a long list of Lisps.

As I understand it, macros can be used to make one
lisp variation act like another.

Given that, the conculsion I infer is that different, very
smart people would use macros to make Python more
"right", even though there's different perceptions of
what is "right."

> A few days ago I tested out a C++ library. It didn't work on the C++
> system I had handy because the STL implementation was
> different/template support was different. etc. etc.

Did you really or are your making that up for the
sake of rhetoric?

> A few days ago I tested out a Python library. It didn't work on the
> implementation I had handy because it was Jython.

Ditto.

That's not to say you couldn't have those problems.  I just
don't like you making up a response in the face of something
what did actually occur.

If it takes more than four decades for different Lisp implementations
to agree on how to import a module, then I think there's
a problem.  And I believe that that problem is that too
many peole who make a Lisp do it because they have a
feeling they know what is right vs. wrong, and emphasize
boosting personal abilities over group ones, and that
tendency is magnified by the inclusion of macros.

> So, what you are saying is that faced with the alternatives of
>
> a) Tweaking an existing, feature rich, mature, proven language, to
>    move it "closer" to your domain.
>
> b) Implementing a new language from scratch, for use in a single
>    domain
>
> you would choose the latter?

False dichotomy.

I needed to evalute a user-defined expression where the
variable names are computed based on calling an associated
function.  The functions may take a long time to compute and
most names are not used in an expression, so I want to
compute the names only when used.

This is different from Python because Python's exec wants
all the variable names defined first.  Hence, this is a new language.

Did I start from scratch?  No!  I used Python to build the
parse tree then tweaked a few nodes of that tree to change
the name lookup into the right form, then generated the
function from that parse tree.

The syntax was the same as Python's, but the behaviour different.
Though back at the Python level, it's a "call this function to get
the needed result", and has precisely the same nature as
any other Python function would have.

> If so, you are choosing the path which pretty much guarantees that
> your software will take much longer to write, and that it will be a
> lot buggier.

Indeed.  And that would be silly.  Luckily, you didn't think
of all the alternatives available to me.

And some languages are not-at-all close to Python.  Eg, I wanted
to implement a domain-specific language called MCL using my
PyDaylight package.  I ended up writing a parser for MCL and
converting the result into Python code, then exec'ing the Python
code.  I would not want to write a macro for Python to support
MCL natively.  Blech!

Similarly, SMILES is a language for describing molecules.  It
isn't a programming language at all, so I just write a parser for it.

I once made a language for manipulating molecules.  It was a
a simple command language with syntax like
  load "abc.pdb" into a
  select "resname LYS" from a into b
  save b as "lysine.pdb"

I wrote it in Perl, but didn't want the language to be part of
Perl (nor of Python).  It wasn't a Perl library because Perl's
syntax was too complicated for the target audience.  It wasn't
very powerful because it only needed to load a molecule,
select portions of the molecule, select subsets, copy, rotate
and translate sets of atoms, and save a set to a file.

Why in the world would I want to extend Lisp/Perl/Python/
whatever to support that language directly?

> It's an extreme form of Greenspunning.

???  Apparantly not Alan Greenspan.

Google yields 4 hits (3 if I exclude your post).  It seems
to be "writing a lot of code yourself instead of using existing
packages"

Then you don't know me works at all and are making
unjustified comments based on false preconceptions.

> How do you reconcile
   ...
> You criticize macros for not encouraging code sharing (they do, by
> encouraging you to share the (vast) underlying language while reaching
> out towards a specific domain), while your preferred solution seems to
> be the ultimate code non-sharing, by throwing away the underlying
> language, and re-doing it.

Because you assume when I say "language" that I mean a powerful
language like Lisp or Python when it's really a domain specific
languages which often aren't even Turing complete or even a programming
language, and which are easy to convert into Python, and designed for
someone with only a little programming experience.

Please tell me how you would implement this language

  load "abc.pdb" into a
  select "resname LYS" from a into b
  save b as "lysine.pdb"

as a macro in Lisp.  I'll assume 'load-pdb' loads a PDB file into
a object which holds a set of atoms, and that object has the
method 'select-atoms' which creates a new (sub)set and also
has the method 'save-as' for saving those atoms in the right format.

And the above is all that the user can type.

How in the world does macros make handling that language
any easier than the standand parser-based non-macro solution?


                    Andrew
                    dalke at dalkescientific.com