question about module compiler: line numbers and positions of substring, which are covered by AST nodes

[Peter Bulychev]

I will answer my own question. Maybe it will be useful for someone else in
future.

Compiler module just transforms parse trees into the abstract syntax trees.
Parse trees are built using Parser module.
Parser module is not pure Python module, it is written in C language and
Python loads it as .so (or .dll) library.
There is a required field in C implementation of parse trees (
node.n_col_offset), but it is unaccessible from Python. Therefore to get the
information about the position of some lexem in the AST built by module
Compiler, you have to perform the following steps:
*) Get the Python sources
*) Modify the Parser module
*) Build Python (or just parser.so)
*) Drag the node.n_col_offset field to the Compiler module

It should be noted that this solution will not be cross-platform.

Good luck!

2008/3/16, Peter Bulychev <peter.bulychev at gmail.com>:
>
> Hello.
>
> I use standard module Compiler to build ASTs of Python code.
>
> Given AST subtree I want to restore coordinates of substring in the input,
> which is covered by the subtree.
>
> Example:
> Suppose, the input is 'a+(b+c)'. It is parsed to the tree ADD(a, ADD(b,c))
> by module Compiler.
> I want to have a function, which given ADD(b,c) will return the
> coordinates of the substring '(b+c)'.
>
> Module Compiler provides only line numbers  for each node. But even this
> information is incorrect: only the first line of multi-lined statements is
> stored.
>
> What is the easiest way of doing what I want?
> As I understand, module Compiler is automatically generated by some parser
> generator. Maybe I can modify it?
>
> --
> Best regards,
>    Peter Bulychev.




-- 
Best regards,
   Peter Bulychev.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20080317/b087d007/attachment-0001.html>