[Patches] [ python-Patches-1212837 ] Improved profiler

[SourceForge.net]

Patches item #1212837, was opened at 2005-06-01 18:05
Message generated for change (Comment added) made by lcreighton
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1212837&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Modules
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Brett Rosen (bdrosen)
Assigned to: Nobody/Anonymous (nobody)
Summary: Improved profiler

Initial Comment:
I tried using both hotshot and the old python profiler
and found them both to be inadequate.

The old python profiler seems to give accurate results
for timing, but is slow enough that it adds significant
overhead to what it is measuing (approximately 10 times). 
Furthermore, it has no ability to give detailed stats
about 
children. (how much of the cumulative time was taken up
by each function called by a function)

The hotshot profiler is much faster (profiling) adding
only 30% overhead. However, it is extremely slow to load
the results from the log file. It does not currently
support detailed child stats, although I imagine that
it could be made to do so using the information in the
log file. The biggest problem with it, is that the
time results seem to be highly inaccurate. (doesn't
correspond to actual seconds, although they seem to be 
proportionally wrong)

To address these shortcomings, I wrote a new profiling
module. It adds about the same overhead (30%) as hotshot,
but is much faster in retrieving results. It supports
detailed child stats and gives accurate timing information
in milliseconds. The accompanying .py module could use
additional work though - because of the child stats, I
was not able to reuse the stats module like hotshot does.

I've included a simple test script that runs pystone
for all 3 profilers (and without the profiler) to give
a better idea of the differences. I've also included
a dump of the output of the script running under Windows
XP with python 2.4.1

----------------------------------------------------------------------

Comment By: Laura Creighton (lcreighton)
Date: 2005-09-21 17:18

Message:
Logged In: YES 
user_id=376262

We've been using Hotshot at Strakt for a while now.  We
tried your patch and
it worked precisely as advertised.  Thank you for writing it.

Laura Creighton

----------------------------------------------------------------------

Comment By: Brett Rosen (bdrosen)
Date: 2005-09-08 17:25

Message:
Logged In: YES 
user_id=1289249

I've enclosed an updated version of profile.c that fixes
a problem in the original patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1212837&group_id=5470