cProfile, timed call tree

[dieter]

Nico Schlömer <nico.schloemer at gmail.com> writes:

> From what I understand about the Python profilers, the type of information
> you get from a stats object is
>
>   * How much time was spent in function X,
>   * what the callers and callees of function X are, and
>   * and bunch of meta info about function X.
>
> With the program
> ```
> def prime(n):
>     # compute the n-th prime number, takes longer for larger n
>     return 2
>
> def a():
>     return prime(1)
>
> def b():
>     return prime(4000)
>
> a()
> b()
> ```
> I would be able to find that `prime` took a lot of time, but not that it
> took more time when called from `b()`. In other words: I cannot construct
> the call tree with times from Stats.

You will see that "prime" was called both from "a" and "b" - and
*in this particular case*, you will see that the execution times
of "b" and "prime" are almost identical and derive that the "prime"
call of b was the expensive one.

But, in general, you are right: you cannot reconstruct complete
call trees. The reason is quite simple: maintaining information
for the complete caller ancestry (rather than just the immediate
caller) is expensive (both in terms of runtime and storage).
Profiling usually is used as a preparation for optimization.
Optimization has the greatest effects if applied to inner loops.
And for the analysis of inner loops, complete call tree information
is not necessary.

> Is this correct? If not, how to get a timed call tree from Stats? Perhaps
> that's not the right think to work with?

You will need to write your own profiler - one that keeps track
not only about the immediate caller of a function call but
of the whole caller ancestry (maybe recursion reduced).

You can see an example of a custom profiler
in "Products.ZopeProfiler" (--> PyPI).
It does not take into account the whole caller ancestry.
Instead it records additional information concerning higher level
Zope (a Web application framework) calls.