[Python-Dev] Move encoding_decl to the top of Grammar/Grammar?
Thomas Lee
tom at vector-seven.com
Tue Dec 2 07:57:11 CET 2008
Hi all,
Currently, Parser/parsetok.c has a dependency on graminit.h. This can
cause headaches when rebuilding after adding new syntax to
Grammar/Grammar because parsetok.c is part of pgen, which is responsible
for *generating* graminit.h.
This circular dependency can result in parsetok.c using a different
value for encoding_decl to what is used in ast.c, which causes
PyAST_FromNode to fall over at runtime. It effectively looks something
like this:
* Grammar/Grammar is modified
* build begins -- pgen compiles, parsetok.c uses encoding_decl=X
* graminit.h is rebuilt with encoding_decl=Y
* ast.c is compiled using encoding_decl=Y
* when python runs, parsetok() emits encoding_decl nodes that
PyAST_FromNode can't recognize:
SystemError: invalid node XXX for PyAST_FromNode
A nice, easy short term solution that doesn't require unwinding this
dependency would be to simply move encoding_decl to the top of
Grammar/Grammar and add a big warning noting that it needs to come
before everything else. This will help to ensure its value never changes
when syntax is added/removed.
I'm happy to provide a patch for this (including some additional
dependency info for files dependent upon graminit.h and Python-ast.h),
but was wondering if there were any opinions about how this should be
resolved.
Cheers,
Tom
More information about the Python-Dev
mailing list