Using graphviz to visualize trace.py output, anybody?

[Magnus Lycka]

svenn.are at bjerkem.de wrote:
> Hi,
> 
> has anybody thought of / already used graphviz to convert the output of
> trace.py into a graph? 

Thanks for the pointer to trace.py. I hadn't seen that before.

Are you thinking about a call-graph or sequence diagram, based on
the output from trace.py --trace? I suspect it might be easiest to
do that by modifying trace.py, so that it gives clearer hints about
when it performs a call, and when it returns from a call. (Could
it do that? Does trace.py have a clue about that?)

E.g. from running trace.py --trace trace.py in Python 2.3 I get:

...
  --- modulename: trace, funcname: ?
...
trace.py(104): PRAGMA_NOCOVER = "#pragma NO COVER"
trace.py(107): rx_blank = re.compile(r'^\s*(#.*)?$')
  --- modulename: sre, funcname: compile
sre.py(179):     return _compile(pattern, flags)
  --- modulename: sre, funcname: _compile
sre.py(218):     cachekey = (type(key[0]),) + key
sre.py(219):     p = _cache.get(cachekey)
sre.py(220):     if p is not None:
sre.py(221):         return p
trace.py(109): class Ignore:
  --- modulename: trace, funcname: Ignore
trace.py(109): class Ignore:
...

That should cause something like:

trace.GLOBAL
  |->sre.compile
  |   |->sre._compile
  |->trace.Ignore

Finding the relevant nodes in the graph seems trivial. It's
the ' --- modulename:...' rows. Figuring out the origin of
the edges to these nodes seems error-prone. How do you know
that the above isn't really...

trace.GLOBAL
  |->sre.compile
      |->sre._compile
          |->trace.Ignore

...for instance? Looking at the output, we can see that it's
not, and we know that sre won't call trace in _compile, but I'm
not sure how to write a parser that would detect all such cases.
I imagine that loops, callbacks etc might turn out to be tricky.

Both calls and returns can look very different in Python, the
program flow can move back and forth a lot, both modules and
callables might have a different name than the one that appears
in the source code where it's called (e.g. re => sre above) etc.

I wrote somthing like this to analyze COBOL source code about a
year ago, and that was trivial, but Python is so dynamic and
varied compared to COBOL where I only had to look for "CALL
something" if my memory serves me right.

Perhaps you should make a simple parser that does the very easy
part, essentially something like:

? -> sre.compile;
? -> sre._compile;
? -> trace.Ignore;

Then you can massage the resulting dot-files manually until they
give you the output you want. You should really do this with non-
trivial code of the kind you would like to analyze, and make sure
that you are able to create meaningful graphs.

For instance, you probably want to show something other than just
the output of...

trace.__main__ -> sre.compile;
sre.compile -> sre._compile;
trace.__main__ -> trace.Ignore;

...since this wouldn't give you any meaningful image of the sequence.
Or perhaps you just want collaboration diagrams?

Actually, even if you end up doing manual work with every dot-file,
it might turn out to be less work that other ways of doing graphs,
at least if you can get typical edges right in your program and just
fix occational ambiguities. I'm not so sure that it's so easy to
get any pretty graphs at all from this approach though. I guess it
requires some experimentation.