Python's parser.

[Michael Hudson]

wtanksle at dolphin.openprojects.net (William Tanksley) writes:

> On 10 May 2000 18:53:22 +0100, Michael Hudson wrote:
> >I thought I might waste some time by having a crack at implementing +=
> >and friends in Python.  I think I know what I want to do on the
> >codegen side, but at the moment that hurdle looks some way off.
> 
> Sounds like fun.
> 
> >How much effort is involved to get Python to accept a '*=' token?
> 
> On the code side, you seem to be developing an accurate idea.  On the
> social side, you can either rework all of Python's smeantics so that +=
> and so on makes consistent sense, or you can kill Guido and use his time
> machine to take his place (careful!  Killing people who have a time
> machine is VERY unlikely to work!).

This is where I have my Idea.  We shall see.  But on one hand, I don't
really care what happens to my mods; I'm doing this mainly for fun,
and maybe preparation for more serious grammar hacking.

I've also sorted out my problem...

> I love this newsgroup -- violence is a solution to every problem.

Yep.

> >at the end of Grammar/Grammar, and changed
> >small_stmt: expr_stmt | print_stmt  | del_stmt | pass_stmt 
> >          | flow_stmt | import_stmt | global_stmt | exec_stmt 
> >          | assert_stmt
> 
> >to
> 
> >small_stmt: expr_stmt | aexpr_stmt | print_stmt | del_stmt | pass_stmt
> >          | flow_stmt | import_stmt | global_stmt | exec_stmt 
> >          | assert_stmt

which made the grammar ambiguous (gotta love accurate error messages).

> This looks okay in and of itself, although I don't have the source code to
> check.  One warning light is staying on, though: I'm surprised that you
> didn't place your assignment statement alongside the other assignment
> statement.

The reason I didn't do that was I didn't want to work out what

a *= b *= 2

meant (or implement it if I did), so I was going to render it illegal.
I've now modified the expr_stmt rule, and will ban multiple
assignments in Python/compile.c.  I think.

[snip]
> You're leaving something out here -- these two snippets of code appear
> identical.  This is also not the right place to make this modifiation,
> although it's right nearby.

Oops.  Cut and paste error. <blush/>.
 
> >Help?  Anyone?  This can't be *that* hard, can it?
> 
> It is.  Easily.  This is why the canonical book on parsing has a picture
> of a dragon on the cover.  Good luck.

The best comment in the Python source is from Parser/pgen.c:

  /* This algorithm is from a book written before the invention of
     structured programming... */

which isn't too encouraging...

Cheers,
M.

-- 
39. Re graphics:  A picture is worth 10K  words - but only those 
    to describe the picture. Hardly any sets of 10K words can be 
    adequately described with pictures.
     -- Alan Perlis, http://www.cs.yale.edu/~perlis-alan/quotes.html