Any way to use cProfile in a multi-threaded program?

[Jonathan Hartley]

On Dec 30, 3:00 pm, mk <mrk... at gmail.com> wrote:
> I'm stumped; I read somewhere that one would have to modify Thread.run()
>   method but I have never modified Python methods nor would I really
> want to do it.
>
> Is there any way to start cProfile on each thread and then combine the
> stats?
>
> Regards,
> mk


Well, you can start cProfile from any arbitrary point within your
program, using the code:

  import cProfile
  import .mymodule
  command = 'mymodule.myfunction()'
  cProfile.runctx(command, globals(), locals(),
filename='profile.out')

Conventionally this is run at the start of your program (and myfunction
() is something that you call to startup the remainder of your
program), but one solution for you might be to run this after the
threads have been created. You'll have to run this in each thread, and
myfunction() will be the function required for the thread to do its
work. Maybe each thread should output to a different filename, using
something like filename='profile%s' % (threading.current_thread
().ident,)

As for combining the outputs from each thread, I don't know what the
cProfile output format is, so I can't help you there.

There might be better ways, but I don't know them.