Python lint tool (was: Re: [Python-Dev] Python keywords (was Lockstep iteration - eureka!))

[James C. Ahlstrom]

Trent Mick wrote:
> 
> On Mon, Aug 14, 2000 at 11:25:59AM -0400, James C. Ahlstrom wrote:
> > What about using Bison/Yacc?  I have been playing with a
> > lint tool for Python, and have been using it.
> >
> Oh yeah? What does the linter check? I would be interested in seeing that.

Actually I have better luck parsing Python than linting it.  My
initial naive approach using C-language wisdom such as checking for
line numbers where variables are set/used failed.  I now feel that
a Python lint tool must either use complete data flow analysis
(hard) or must actually interpret the code as Python does (hard).
All I can really do so far is get and check function signatures.
I can supply more details if you want, but remember it doesn't
work yet, and I may not have time to complete it.  I learned a
lot though.

To parse Python I first use Parser/tokenizer.c to return tokens,
then a Yacc grammar file.  This parses all of Lib/*.py in less
than two seconds on a modest machine.  The tokens returned by
tokenizer.c must be massaged a bit to be suitable for Yacc, but
nothing major.

All the Yacc actions are calls to Python methods, so the real
work is written in Python.  Yacc just represents the grammar.

The problem I have with the current grammar is the large number
of confusing shifts required.  The grammar can't specify operator
precedence, so it uses shift/reduce conflicts instead.  Yacc
eliminates this problem.

JimA