[Python-Dev] Standardise the AST (Re: PEP 563: Postponed Evaluation of Annotations)

[Nick Coghlan]

On 13 November 2017 at 10:26, Guido van Rossum <guido at python.org> wrote:
> [Greg]
>> Proponents of Lisp point to the advantages of easily
>> being able to express Lisp programs using Lisp data
>> structures. There would also be benefits in having a
>> standard way to represent Python programs using Python
>> data structures.
>
> But we have to weigh the advantages against other requirements. IIRC Lisp
> had almost no syntax so I presume the mapping to data structures was nearly
> trivial compared to Python.

As far as I recall, the base primitives in Lisp are something like
UnaryOp, BinOp, and an asymmetric head/tail BinOp variant for working
with sequences.

That said, I think it could be genuinely useful to formally define a
"Simple Python expression" concept that:

- allowed symbolic Python expressions (including literals & displays)
- prohibited the use of keywords (avoiding the harder cases like
lambda, yield, await, and comprehensions)
- allowed name references (but didn't resolve them at compile time)
- defined an "escape prefix" to explicitly indicate references to
Python globals & builtins

That specific set of characteristics is drawn from the syntax used for
queries on pandas data frames:
https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.query.html

Right now, the data frame API relies on strings for that capability,
and pandas.eval to do the evaluation.

pandas.eval in turn provides two levels of configurability:

* which parser/compiler to use (which affects operand precedence)
* which execution engine to use (numexpr is much faster for SciPy
components than a regular Python eval)

For an integrated-into-Python variant, we presumably wouldn't allow
configurable operand precedence (we'd use the same rules as normal
expressions), but we could still offer a runtime expression type that
was compiled at the same time as everything else, but rather than
accepting parameters like a regular function, instead accepted a
namespace to use when evaluating the expression.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia