[Python-Dev] cProfile and threads

[Nick Coghlan]

2010/8/17 Kristján Valur Jónsson <kristjan at ccpgames.com>:
> Hello there.
>
> I‘d like to draw your attention to two feature requests / patches that I‘ve
> subbmitted:
>
> http://bugs.python.org/issue9609
> http://bugs.python.org/issue9622
>
> These patches are the result of work that we have done in profiling
> Stackless Python server applications at runtime, but they apply just as well
> to C Python.

Both look like good ideas to me (multi-threaded profiling and
debugging is fairly painful and it would be good to be able to do
something to improve that situation).

> The first patch makes _lsprof, the engine behind cProfile, multi-stack
> aware.  This allows the same cProfiler.Profile() instance to be active on
> multiple python threads and still meaningful information is gathered.

I'm curious as to the memory impact this has on the profiler (it
obviously can't be too bad if you're able to run it against your live
servers).

> The second patch allows to set the trace/profile function in python
> globally, so that all threads are affected.  This is essential if you want
> to take a profililng snapshot of a running application.

One minor quibble here is that I would suggest using "global=False" in
your docstring signatures.

Both patches seem to be missing updates to the relevant documentation.
I expect this would be difficult to unit test properly, but at least
some basic tests to check that the new global configuration of
settrace and setprofile don't blow, and that a profiler can be shared
between two threads would be good.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia