parsing python code

[Kay Schluehr]

On Dec 5, 6:02 pm, "Ryan Krauss" <ryanli... at gmail.com> wrote:
> I need to parse a Python file by breaking it into blocks matching
> indentation levels so that function definitions, for loops, and
> classes are kept together as blocks.  For example, if I have something
> like
>
> from scipy import*
> from pylab import*
>
> g = .6
> Input_freq = 10.0
>
> def load_data(path):
>         data = loadtxt(path, skiprows = 1)
>         t = data[:,0]
>         Input = data[:,1]
>         Output = data[:,2]
>         return t,Input,Output
>
> def time_plot(x,y,n = 1):
>         figure(n)
>         clf()
>         for curx, cury in zip(x,y):
>                 plot(curx,cury)
>         title('Time Plot')
>         xlabel('time')
>         ylabel('f(t)')
>         legend(['Input','Output'],1)
>         return figure(n)
>
> t, Input, Output = load_data('system_data.txt')
>
> I would like the blocks to be
>
> block1 = ['from scipy import*', 'from pylab import*', 'g =
> .6','Input_freq = 10.0']
>
> block2 = ['def load_data(path):',
> '       data = loadtxt(path, skiprows = 1)',
> '       t = data[:,0]',
> '       Input = data[:,1]',
> '       Output = data[:,2]',
> '       return t,Input,Output']
>
> and so on.
>
> I think the parser module should enable me to do this, but I can't
> seem to figure it out.  Specifically, I think I need to use
> parser.sequence2ast, but it doesn't work the way I think it should and
> I can't find more documentation on it or an example that uses it.
>
> I tried
>
> f = open('Example.py','r')
> mylines = f.readlines()
> parser.sequence2ast(mylines)
>
> but got a ParserError.
>
> Is there an easy way to do what I need?
>
> Thanks,
>
> Ryan

You have to use parser.suite:

f_content = open('Example.py').read()
_ast = parser.suite(f_content)

You can display the _ast object as a nested list:

_lst = _ast.tolist()

This list can be converted back to an ast object using sequence2ast.

Another option is to use parser.expr instead of parser.suite but this
only applies for Python expressions but not statements. So
parser.expr / parser.suite are like eval / exec.