Usage of ast.

[Chris Angelico]

On Tue, Feb 28, 2017 at 3:17 AM, Jon Ribbens <jon+usenet at unequivocal.eu> wrote:
> On 2017-02-27, Chris Angelico <rosuav at gmail.com> wrote:
>> Actually it does execute, as you can see from the source code.
>
> I'm not sure what you mean by that. I was looking at the source code
> (and its history) when I wrote my post, and I would be hard pushed to
> describe it as "executing" anything.

You could say that it "interprets" the AST. It has exactly the same
kind of if/elif tree of "this kind of thing? do this" that a naive and
unoptimized language interpreter would use. Does that not count as
execution?

>>>>> ast.literal_eval("[1,2,3]")
>> [1, 2, 3]
>>
>> This isn't a literal, and it's evaluated by literal_eval.
>
> Actually, that is a literal, in the computer science sense: "a literal
> is a notation for representing a fixed value in source code".
> I realise it isn't a literal in the sense of "as defined by section
> 2.4 of the Python Language Reference", but then perhaps most people
> would expect the word to have its more generic meaning.

So what _is_ a literal? Can you have a literal that evaluates to a
non-constant, or a non-literal that gets compiled to a constant? Here
are examples of both:

>>> dis.dis(lambda: [1,2,3])
  1           0 LOAD_CONST               1 (1)
              2 LOAD_CONST               2 (2)
              4 LOAD_CONST               3 (3)
              6 BUILD_LIST               3
              8 RETURN_VALUE

By your definition, this is a literal, but it most surely does not
represent a fixed value. Every time you call this function, it will
construct and return a brand new list.

>>> dis.dis(lambda x: x in {1,2,3})
  1           0 LOAD_FAST                0 (x)
              2 LOAD_CONST               4 (frozenset({1, 2, 3}))
              4 COMPARE_OP               6 (in)
              6 RETURN_VALUE

There's no literal syntax "frozenset({1, 2, 3})", but that got loaded
as a constant.

> Python doesn't seem to have a word for "[1, 2, 3]", inasmuch as
> it is an example of a "list display", but that form is only a subset
> of what "list display" means (displays include comprehensions), so
> it is not true to say that literal_eval() can evaluate displays.
> If we had to choose a way to describe it, it seems unlikely that
> there's a better description than "list literal".

It's a list display.

> (See also the "What's new" announcement for Python 3.2 where the
> addition of set support to literal_eval() was described as adding
> "set literals". https://docs.python.org/3/whatsnew/3.2.html#ast )

Technically they're called "set display", too. Does it count as a
literal if and only if all members are literals? That's a pretty
tricky concept to try to bake into a definition, but that's exactly
the recursive definition that literal_eval actually uses.

>> It's basically a "safe eval of simple expressions".
>
> What I was trying to say was that I think that is a very misleading
> way of describing it. It is not entirely untrue, but it is much closer
> to being false than it is to being true. It isn't "safe eval", and it
> is not intended to deal with any expressions.
>
> (See also http://bugs.python.org/issue22525 )

Given the many subtle differences in usage of the word "literal", I'm
happy with it using that name. It's not 100% accurate, but it covers
the non-pedantic uses of the term. Its docstring most definitely says
"expression", though, because there are some non-literal expressions
that it DOES evaluate, and for good reason.

ChrisA