[Python-ideas] History on proposals for Macros?

[Andrew Barnert]

On Mar 27, 2015, at 16:23, Carl Meyer <carl at oddbird.net> wrote:
> 
>> On 03/27/2015 05:19 PM, Matthew Rocklin wrote:
>> I want Python to have macros.  This is obviously a hard sell.  I'm
>> willing to do some legwork to demonstrate value.
>> 
>> What would a good proposal include?  Are there good examples of failed
>> proposals on this topic?
>> 
>> Is the core team open to this topic?
> 
> You probably want to have a look at MacroPy [1].
> 
> I don't think it was ever seriously proposed for core, though.

I think it might be worth looking at what smaller core changes could allow MacroPy to be better or simpler than it is. Some of the changes to the import mechanism and the ast module that we've already had between 3.0 and 3.4 have done that, and also have the benefit of making a Python implementation easier to understand and to hack on. I suspect the same would be true for other possible changes. And then, once those changes are in and MacroPy is as simple, pleasant, and wart-free as it can be, it would probably be an easier sell (although still maybe not easy...) to get something like it incorporated into the core compiler and/or importer instead of as an external import hook.

Of course you may look at MacroPy and think, "This is way too big and complicated, I just want to be able to do what Language X had without learning all this other stuff", but then it'll at least help focus the ideas on what you do and don't want, right?

Anyway, I think people who proposed macros in the past have had two major problems.

First, they didn't think through the design. Lisp-style macros don't work in a language with syntax more complex than s-expressions. And making them partially text-based doesn't help either, given the way whitespace works in Python. And once you get to the point where you're doing AST transforms with the ast module, that doesn't really feel like macros anymore. If you instead come at it from the perspective of Dylan or one of the ML-with-macros variants, where what you're looking for is a way to write AST transformers in a declarative language that fits into the host language, you might get a lot father. 

Second, they didn't think through the marketing. Ask any famous Lispers why macros are important, and the answer is that they let you reprogram the language first, into a language that's easier to write your actual program in. That's a blatantly anti-Pythonic thing to do. You often can't read good Lisp code until you first learn the new, project-specific language they've built on Lisp, and nobody wants that for Python. Even smaller versions of that, like being able to create new flow control syntax, etc. are an anti-selling-point for Python. So, you need to think of a new way to sell macros that explicitly disavows the idea of creating new syntax, but still shows how macros can usefully do things that look and feel Pythonic but can't be done in Python.