Python language spec

[Neel Krishnaswami]

Nolan Darilek <nolan_d at bigfoot.com> wrote:
> 
> Here's where my confusion lies. Maybe I'm studying the language
> reference and taking it too literally. I'm working slowly through,
> section-by-section, writing rules for each construct and testing
> them.

If what you are really interested in is the grammar, you should really
start with the Grammar file in the Python source distribution. It
defines the grammar of the language in a short, succint file. Save the
language reference for the second phase -- once you have a vague idea
of what all the pieces of the grammar are you might be less distracted
by the minutiae of the grammar definition.

> I'm confused by the expression_list definition, however. It reads:
> 
> expression_list:      expression ("," expression)* [","]
>
> I understand the syntax of the definition, but I don't know what the
> syntax for an expression should be. Earlier rules define u_expr,
> a_expr, etc. but not a regular expression (I don't mean regexp, just a
> no-frills expression :). The only rule which defines 'expression' is:

Here's the definition of expression from the grammar file, plus some
more productions that it references.

  expr: xor_expr ('|' xor_expr)*
  xor_expr: and_expr ('^' and_expr)*
  and_expr: shift_expr ('&' shift_expr)*
  shift_expr: arith_expr (('<<'|'>>') arith_expr)*
  arith_expr: term (('+'|'-') term)*
  term: factor (('*'|'/'|'%') factor)*
  factor: ('+'|'-'|'~') factor | power
  power: atom trailer* ('**' factor)*
  atom: '(' [testlist] ')' | '[' [testlist] ']' | '{' [dictmaker] '}' |
  	'`' testlist '`' | NAME | NUMBER | STRING+

Now, a reasonable question to ask is: why are there so many different
kinds of expression when they all seem to reduce to the same thing? 
(This seemed to me to be the thing that was confusing you. Let me know
if I misunderstood.)

The answer is that different operators have different precedences, and
having separate rules is a way of ensuring that the parser will emit a
syntax tree that properly observes the operator precedences that are
expected.  A simple way of getting an intuition about this is to write
down some arithmetic expressions and then manually reduce them using
the rules of the grammar.

For example, try parsing the expressions "0 & 0 | 1" and "3 + 5 * 7".


Neel