your mail

[Leonardo B. Lopes]

Dear Skip,

	Thanks! I think you've found the crux of the problem. Do you mind
if I forward this message to the two groups? I think it would really help
the other people on the threads. So if I understand correctly, you are
right and the profiler is not measuring elapsed time but indeed only the
cpu time used by the python process. So if it is blocked waiting for the
mysql process, the time of the query is not taken into account. That would
explain why the "easy" query and the "hard" query take about the same time
according to the profiler. Is that precise?

Leo.

On Thu, 4 Oct 2001, Skip Montanaro wrote:

> 
>     Leonardo> Thanks for the tip. I actually have used the explain
>     Leonardo> statement. But are you sure about the profiler ignoring the
>     Leonardo> time taken by the query? Why would it be excluded from the
>     Leonardo> computation time?
> 
> If the profiler is measuring cpu seconds, it will be measuring the
> computation time of the Python process.  The computation time of other
> processes involved in the computation (possibly on other machines) just
> isn't available to it.  If it's measuring elapsed (wall clock) time, then
> sure, it gives you some measure of the time it takes for the entire
> computation to complete.  However, that would include networking delays as
> well as delays caused by either or both the Python program and the MySQL
> database server not actually having the CPU.
> 
>     Leonardo> How would the profiler even be able to distinguish between
>     Leonardo> time waiting in a function for some external process and time
>     Leonardo> spent in the python interpreter itself?
> 
> The profiler uses time.clock() if it's available, which is generally true on
> most Unix-like systems.  Again, on Unix-like systems, time.clock() returns
> the CPU time of the current process, not the elapsed time.  From the clock()
> man page:
> 
>     DESCRIPTION
>            The clock() function returns an approximation of processor
>            time used by the program.
> 
>     Leonardo> I checked the manual for the profiler section. It says that
>     Leonardo> the profiler is activated by callbacks when functions are
>     Leonardo> called and when they return, so if a function is blocked
>     Leonardo> waiting for mysql, that time should be computed.
> 
> It all depends.  Take a look at the code in the Profile class's __init__
> method:
> 
>     if not timer:
>         if os.name == 'mac':
>             self.timer = MacOS.GetTicks
>             self.dispatcher = self.trace_dispatch_mac
>             self.get_time = _get_time_mac
>         elif hasattr(time, 'clock'):
>             self.timer = self.get_time = time.clock
>             self.dispatcher = self.trace_dispatch_i
>         elif hasattr(os, 'times'):
>             self.timer = os.times
>             self.dispatcher = self.trace_dispatch
>             self.get_time = _get_time_times
>         else:
>             self.timer = self.get_time = time.time
>             self.dispatcher = self.trace_dispatch_i
> 
> If you're running on Windows (I think you said something about Linux), then
> time.clock is either unavailable or is, but doesn't record CPU time.  On
> Linux systems it does (though to varying degrees of (in)accuracy).
> 
> -- 
> Skip Montanaro (skip at pobox.com)
> http://www.mojam.com/
> http://www.musi-cal.com/
> 

========================================================================
Leonardo B. Lopes                                       leo at iems.nwu.edu 
Ph.D. Student                                              (847)491-8470
IEMS - Northwestern University              http://www.iems.nwu.edu/~leo