[Python-Dev] Standardise the AST (Re: PEP 563: Postponed Evaluation of Annotations)

[Guido van Rossum]

On Sun, Nov 12, 2017 at 2:39 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>
wrote:

> Guido van Rossum wrote:
>
>> The PEP answers that clearly (under Implementation):
>>
>>  > If an annotation was already a string, this string is preserved
>>  > verbatim.
>>
>
> This bothers me, because it means the transformation from
> what you write in the source and the object you get at
> run time is not reversible. Something interpreting the
> annotations as Python expressions at run time has no way
> to know whether a Python expression was written in the
> source or a string literal whose contents happen to look
> like an expression.
>

The rule is a form of normalization. It is the best way to ensure that the
following two will mean the same thing:

def foo(a: int): ...
def foo(a: 'int'): ...

To reverse the transformation you have two options: always generate string
literals, or always generate non-string annotations. The resulting code
could look like either of the above two, and the meaning should be
identical. (There are edge cases, e.g.:

def foo(a: 'int)'): ...
def foo(a: '"int"'): ...

but they aren't currently valid as  PEP 484 type annotations and it's no
burden to keep them out of future syntactic extensions of annotations.)


> I still think that the run-time form of a non-evaluated
> annotation should be some form of AST. That's been rejected
> on the grounds that the structure of an AST is considered
> an implementation detail that can change between Python
> versions.
>
> However, that in itself seems like a bad thing to me.
> There *should* be a standard and stable form of AST
> provided that doesn't change unless the syntax of Python
> changes. It doesn't have to be the same as what's used
> internally by the compiler.
>

But Python's syntax changes in nearly every release. And even though we
nearly always take pains not to invalidate source code that used to be
valid, that's not so easy at the AST level, which elides many details (such
as whitespace and parentheses).


> Proponents of Lisp point to the advantages of easily
> being able to express Lisp programs using Lisp data
> structures. There would also be benefits in having a
> standard way to represent Python programs using Python
> data structures.
>

But we have to weigh the advantages against other requirements. IIRC Lisp
had almost no syntax so I presume the mapping to data structures was nearly
trivial compared to Python.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171112/5990e505/attachment.html>