[Python-ideas] History on proposals for Macros?

[Guido van Rossum]

Matthew,

Is something stopping you from exploring this? Do you have specific ideas
on how to improve on macropy?

It sounds almost as if you would like to implement this but you want some
kind of promise ahead of time that your work will be incorporated into the
language. But that's just not how it works. When you want to explore a big
idea like this, at some point you have to be willing to take the risk of
writing code without a guaranteed pay off. Haoyi didn't ask for macropy to
be incorporated into Python -- in fact he was surprised at the amount of
uptake it got.

You've received quite a bit of feedback (and, may I say, push back :-) from
a small number of python-ideas veterans -- you can take this or leave it,
but at this point I think you've gotten about as much mileage out of the
list as can be expected.

Good luck!

--Guido

On Mon, Mar 30, 2015 at 8:27 PM, Matthew Rocklin <mrocklin at gmail.com> wrote:

> Also, just to credentialize myself, I am not a huge Lisp lover.  I don't
> want macros to do crazy logic programming or whatever.  I write numeric
> code in the scientific Python ecosystem.  I want macros to build better
> interfaces for downstream users.  This seems to be the modern use case in
> user-focused languages rather than lisp-magic-hell.
>
> On Mon, Mar 30, 2015 at 8:20 PM, Matthew Rocklin <mrocklin at gmail.com>
> wrote:
>
>> Lisps like Scheme do indeed have an easier time with these due to the
>> whole code-is-data thing, it's quite doable in languages with real syntax
>> though.  R and Julia would be good examples of syntactic languages with
>> full macros.  The Julia implementation might be a good model for what
>> Python could do.   Their docs
>> <http://julia.readthedocs.org/en/latest/manual/metaprogramming/> are
>> also a nice read if anyone isn't familiar with the topic.
>>
>> Macropy represents unevaluated expressions with the objects from the ast
>> module.  This seems like a sane choice.
>>
>> To be a little pedantic I'll give a brief example loosely showing what a
>> macro is, then I'll talk about assert statements as a use case where macros
>> might help with a pain point in normal Python programming.
>>
>> *Brief Educational Blurb*
>>
>> We write code as text
>>
>> defmacro f(x):
>>     ...
>>
>> f(a + b) * sin(c)
>>
>> We then parse parts of that text into syntax trees.
>>
>> [image: Inline image 1]
>> Usually we translate these trees into byte-code and evaluate bottom-up,
>> starting with a + b, then applying f, etc... Macros stop this process.
>> They capture the subtrees beneath them before execution.  Whenever we see a
>> macro (f), we don't evaluate its subtree (a + b).  Instead we transform
>> the subtree into an in-memory representation (perhaps ast.BinOp(a,
>> ast.Add(), b)) and hand that to f to do with as it will.  Lets see an
>> example with assertions.
>>
>> *Use case with Assertions*
>>
>> When testing we often want to write statements like the following
>>
>> assert x == y
>> assert x in y
>> etc...
>>
>> When these statements fail we want to emit statements that are well
>> informed of the full expression, e.g.
>>
>>     5 != 6
>>     5 was not found in {1, 2, 3}
>>
>> In Python we can't do this; assert only gets True or False and doesn't
>> understand what generated that value .  We've come up with a couple of
>> workarounds.  The first is the venerable unittest.TestCase methods that
>> take the two sides of the comparison explicitly e.g. assertEquals(a, b),
>> assertContains(a, b).  This was sufficiently uncomfortable that projects
>> like py.test arose and gained adoption.  Py.test goes through the trouble
>> of parsing the python test_.py files in order to generate nicer error
>> messages.
>>
>> Having macros around would allow users to write this kind of
>> functionality directly in Python rather than resorting to full text parsing
>> and code transformation.  Macros provide an escape out of pure bottom-up
>> evaluation.
>>
>> On Sun, Mar 29, 2015 at 5:53 PM, Guido van Rossum <guido at python.org>
>> wrote:
>>
>>> On Sat, Mar 28, 2015 at 9:53 AM, Matthew Rocklin <mrocklin at gmail.com>
>>> wrote:
>>>
>>>> Responding to comments off list:
>>>>
>>>> I'm not referring to C-style preprocessor macros, I'm referring to
>>>> macros historically found in functional languages and commonly found in
>>>> many user-targeted languages built in the last few years.
>>>>
>>>
>>> Do you have examples and references? IIRC there's something named macros
>>> in Scheme but Scheme, unlike Python, completely unifies code and data, and
>>> there is a standard in-memory representation for code.
>>>
>>>
>>>> The goal is to create things that look like functions but have access
>>>> to the expression that was passed in.
>>>>
>>>> Some examples where this is useful:
>>>>
>>>>     plot(year, miles / gallon)  # Plot with labels determined by
>>>> input-expressions, e.g. miles/gallon
>>>>
>>>>     assertRaises(ZeroDivisionError, 1/0)  # Evaluate the rhs 1/0 within
>>>> assertRaises function, not before
>>>>
>>>>     run_concurrently(f(x), f(y), f(z))  # Run f three times in three
>>>> threads controlled by run_concurrently
>>>>
>>>> Generally one constructs something that looks like a function but,
>>>> rather than receiving a pre-evaluated input, receives a syntax tree along
>>>> with the associated context.  This allows that function-like-thing to
>>>> manipulate the expression and to control the context in which the
>>>> evaluation occurs.
>>>>
>>>
>>> None of the examples need the syntax tree though. The first wants the
>>> string, the last probably just want a way to turn an argument into a lambda.
>>>
>>>
>>>> There are lots of arguments against this, mostly focused around
>>>> potential misuse.  I'm looking for history of such arguments and for a
>>>> general "Yes, this is theoretically possible" or "Not a chance in hell"
>>>> from the community.  Both are fine.
>>>>
>>>
>>> I don't think this is a mainline need in Python, so it's probably both.
>>> :-)
>>>
>>> --
>>> --Guido van Rossum (python.org/~guido)
>>>
>>
>>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150330/3e2e218b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: macro.png
Type: image/png
Size: 17961 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150330/3e2e218b/attachment-0001.png>