PYT - The expressions described in the Python language reference yield only boolean values

[Peter J. Holzer]

On 2022-02-19 23:28:28 +0100, vanyp wrote:
> *I am trying to learn Python from the grammar given in the Python language
> reference and I am surprised.*
> 
> *Lets start here:*
> 
> *"*
> *6.3.4. Calls*
> 
> A call calls a callable object (e.g., a function
> <https://docs.python.org/3/glossary.html#term-function>) with a possibly
> empty series of arguments
> <https://docs.python.org/3/glossary.html#term-argument>:
> 
> *call *::= |primary <https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-primary>|"("
> [|argument_list <https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-argument_list>|[","]
> | |comprehension <https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-comprehension>|]

With all those links this is very hard to read. Please try to format
your mails in a way that makes them easy to follow.

[...][

> *The first or_test is strange, I assume it should be replaced by expression.*

Nope.

> *But even so I think that the only ways out of the recursion are the or_test
> or the lambda_expr.*
> 
> *And by the grammar, those evaluate to booleans as follows:*

No. A grammar specifies how to parse the text, it doesn't say anything
about the types of these expressions (especially not in a dynamically
typed language like python). 

> *Where did I, or the language reference, go wrong?*

First, you neglected the difference between parsing an expression and
evaluating it.

Secondly, you didn't see the way out. 

For example, let's parse the expression

«f(1+2, 3)»

Can this be a call?

To confirm this, is has to match 

    call                 ::=  primary "(" [argument_list [","] | comprehension] ")"

which looks plausible, so let's dive in:

    primary ::=  atom | attributeref | subscription | slicing | call

    atom      ::=  identifier | literal | enclosure

    identifier   ::=  xid_start xid_continue*

«f» is an identier which is an atom which is a primary.

Good so far. Now for the argument-list:

    argument_list        ::=  positional_arguments ["," starred_and_keywords] ...

    positional_arguments ::=  positional_item ("," positional_item)*

We have two comma-separated items, good so far. Check «1+2» first

    positional_item      ::=  assignment_expression | "*" expression

    assignment_expression ::=  [identifier ":="] expression

    expression             ::=  conditional_expression | lambda_expr

    conditional_expression ::=  or_test ["if" or_test "else" expression]

    or_test  ::=  and_test | or_test "or" and_test

    and_test ::=  not_test | and_test "and" not_test

    not_test ::=  comparison | "not" not_test

    comparison    ::=  or_expr (comp_operator or_expr)*

    or_expr  ::=  xor_expr | or_expr "|" xor_expr

    xor_expr ::=  and_expr | xor_expr "^" and_expr

    and_expr ::=  shift_expr | and_expr "&" shift_expr

    shift_expr ::=  a_expr | shift_expr ("<<" | ">>") a_expr

    a_expr ::=  m_expr | a_expr "+" m_expr | a_expr "-" m_expr

Oof. That was a deep recursion, but we finally found a «+». 
So 1 must be an a_expr and 2 an m_expr. Actually recursing once more reveals
that both can be m_expr:

    m_expr ::=  u_expr | m_expr "*" u_expr | m_expr "@" m_expr | ...

    u_expr ::=  power | "-" u_expr | "+" u_expr | "~" u_expr

    power ::=  (await_expr | primary) ["**" u_expr]

    primary ::=  atom | attributeref | subscription | slicing | call

    atom      ::=  identifier | literal | enclosure

    literal ::=  stringliteral | bytesliteral | integer | floatnumber | imagnumber

Ok, they are both integer. 

Then we do basically the same for the second argument and we arrive at
the parse tree:

[warning: fixed width font required]

                                    call
                                      |
   -----------------------------------+--------------------------
  /        /                          |                           \
  |        |                    argument_list                      |
  |        |                    positional_arguments               |
  |        |                   /          |         \              |
  |        |       positional_item        | positional_item        |
  |        |       assignment_expression  | assignment_expression  |
  |        |       expression             | expression             |
  |        |       conditional_expression | conditional_expression |
  |        |       or_test                | or_test                |
  |        |       and_test               | and_test               |
  |        |       not_test               | not_test               |
  |        |       comparison             | comparison             |
  |        |       or_expr                | or_expr                |
  |        |       xor_expr               | xor_expr               |
  |        |       and_expr               | and_expr               |
  |        |       shift_expr             | shift_expr             |
  |        |       a_expr                 | a_expr                 |
  |        |      /  |   \                |                        |
  |        | m_expr  | m_expr             | m_expr                 |
  |        | u_expr  | u_expr             | u_expr                 |
  |        | power   | power              | power                  |
  |        | primary | primary            | primary                |
primary    | atom    | atom               | atom                   |
atom       | literal | literal            | literal                |
identifier | integer | integer            | integer                |
f          ( 1       + 2                  , 3                      )

(Of course, most real parsers start at the bottom and build up, but I
think for humans the top-down approach is more convenient.)
        hp


-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220226/139577ef/attachment.sig>