[Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
SourceForge.net
noreply@sourceforge.net
Fri, 07 Feb 2003 04:08:23 -0800
Bugs item #678518, was opened at 2003-02-01 02:23
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470
Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Michael Hudson (mwh)
Summary: Another parsermodule validation error
Initial Comment:
The following code generates an validation error. It's an
invalid ast tree where the global_stmt doesn't have the
keyword 'global' as a child.
I tracked down the fix to validate_global_stmt() this time.
Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
... [symbol.stmt, [symbol.simple_stmt,
[symbol.small_stmt, [symbol.global_stmt,
[1, 'foo']]], [token.NEWLINE, '']]],
... [token.NEWLINE, ''],
... [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this
to the maintainer!
>>>
----------------------------------------------------------------------
>Comment By: Michael Hudson (mwh)
Date: 2003-02-07 12:08
Message:
Logged In: YES
user_id=6656
Pff, I was right, but logistix's patch fix it.
logistix, did you forget to attach the encoding patch? I
still get errors in module's with a cookie.
----------------------------------------------------------------------
Comment By: Michael Hudson (mwh)
Date: 2003-02-07 11:58
Message:
Logged In: YES
user_id=6656
Patches as such are probably fine. The reason I haven't
checked them in is a wierdness that I may or may not be
related, but is bothering me. Consider:
->> import parser, sys,glob,os, traceback
/>> for path in sys.path:
|.. for fil in glob.glob(os.path.join(path,"*.py")):
|.. try:
|.. try:
|.. a = parser.suite(file(fil).read())
|.. b = parser.ast2tuple(a)
|.. c = parser.sequence2ast(b)
|.. except parser.ParserError:
|.. print "Parse of %s" % fil, "FAILED"
|.. traceback.print_exc()
|.. print '----------------------------'
|.. except:
|.. print "waaa?"
|.. else:
|.. print "what?"
\__
Parse of /home/mwh/src/sf/python/dist/src/Lib/tarfile.py FAILED
Traceback (most recent call last):
File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/heapq.py FAILED
Traceback (most recent call last):
File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/getopt.py FAILED
Traceback (most recent call last):
File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/inspect.py FAILED
Traceback (most recent call last):
File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/pydoc.py FAILED
Traceback (most recent call last):
File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/lib-tk/Tix.py
FAILED
Traceback (most recent call last):
File "<input>", line 7, in ?
ParserError
----------------------------
Traceback (most recent call last):
File "<input>", line 2, in ?
ParserError: Illegal terminal: expected "__assert__"
->>
Notice that an exception seems to have made it through the
except clauses! The scary bit is that a (seemingly) random
number of parser errors *will* caught, and then one will get
through.
The only explanation I have for this is some kind of
returning non-NULL with exception set kind of bug.
Off to play with a debug build, I guess.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 01:17
Message:
Logged In: YES
user_id=33168
Michael were these patches ok to check in?
----------------------------------------------------------------------
Comment By: Michael Hudson (mwh)
Date: 2003-02-03 11:00
Message:
Logged In: YES
user_id=6656
Thanks for doing this. I'll probably check these in after
I've finished crunching my way through monday morning email...
----------------------------------------------------------------------
Comment By: logistix (logistix)
Date: 2003-02-01 21:42
Message:
Logged In: YES
user_id=699438
And I found a few more by doing a roundtrip
ast2Tuple/tuple2ast on all files in pythonpath. These actually
affect valid AST's, so they're probably a bigger deal.
Valid assert statements were failing. The patch was trivial,
so its attached.
Also, files that took advantage of encoding_decl (from PEP
263) were failing. I couldn't tell if PEP 263 was offically
closed, and didn't know what the offical grammar was, so I
didn't do anything there.
I'll also attach the test script that turned up these errors.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470