cProfile, timed call tree

[Nico Schlömer]

Thanks, Dieter, for the concise answer.

Cheers,
Nico

On Sat, May 26, 2018 at 7:42 AM dieter <dieter at handshake.de> wrote:

> Nico Schlömer <nico.schloemer at gmail.com> writes:
>
> > From what I understand about the Python profilers, the type of
> information
> > you get from a stats object is
> >
> >   * How much time was spent in function X,
> >   * what the callers and callees of function X are, and
> >   * and bunch of meta info about function X.
> >
> > With the program
> > ```
> > def prime(n):
> >     # compute the n-th prime number, takes longer for larger n
> >     return 2
> >
> > def a():
> >     return prime(1)
> >
> > def b():
> >     return prime(4000)
> >
> > a()
> > b()
> > ```
> > I would be able to find that `prime` took a lot of time, but not that it
> > took more time when called from `b()`. In other words: I cannot construct
> > the call tree with times from Stats.
>
> You will see that "prime" was called both from "a" and "b" - and
> *in this particular case*, you will see that the execution times
> of "b" and "prime" are almost identical and derive that the "prime"
> call of b was the expensive one.
>
> But, in general, you are right: you cannot reconstruct complete
> call trees. The reason is quite simple: maintaining information
> for the complete caller ancestry (rather than just the immediate
> caller) is expensive (both in terms of runtime and storage).
> Profiling usually is used as a preparation for optimization.
> Optimization has the greatest effects if applied to inner loops.
> And for the analysis of inner loops, complete call tree information
> is not necessary.
>
> > Is this correct? If not, how to get a timed call tree from Stats? Perhaps
> > that's not the right think to work with?
>
> You will need to write your own profiler - one that keeps track
> not only about the immediate caller of a function call but
> of the whole caller ancestry (maybe recursion reduced).
>
> You can see an example of a custom profiler
> in "Products.ZopeProfiler" (--> PyPI).
> It does not take into account the whole caller ancestry.
> Instead it records additional information concerning higher level
> Zope (a Web application framework) calls.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>