[Python-Dev] timeit module

[Nick Coghlan]

Steve Holden wrote:
> Connelly Barnes wrote:
>> Hi,
>>
>> Perhaps I am the only one bothered by the timeit
>> module, but it seems poorly designed to me.
>>
>> First of all, it should use a geometric series with a
>> timeout value to detect how many iterations it should
>> perform.  Currently, the user is required to manually
>> specify the number of iterations (the default is 1
> 
> The provision of a default value generally seems to chop the legs from 
> your argument that the number of iterations is required.

 From the command line, timeit autocalibrates to use the smallest power of ten 
that results in taking at least 0.2 seconds to execute all iterations.

IMO, making this the default behaviour of the timer objects themselves would 
be *much* more useful behaviour than the current default to 10e6 iterations. 
I've been caught a few times hammering on Ctrl-C when I realised that I'd just 
told the interpreter to go do a million iterations of something that took a 
few seconds for each attempt.

>> million).  If the user optimizes his or her code, then
>> the number of iterations must be changed.  If the user
>> moves to a slower or faster computer, then the number
>> of iterations must be changed again.  This is
>> annoying.
>>
> What? The purpose of timeit is to give an approximation to the length of 
> time to run a specific piece ofcode.
> 
> Why must the number of iterations be changed when moving to a slower or 
> faster computer?

Because the default of 10e6 may be entirely inappropriate depending on how 
long an individual iteration is. If a single iteration takes a microsecond, 
great, but if it takes a second, better send out for pizza (or maybe go for a 
trip around the world in the 4 months that sucker is going to take to execute!)

If Timer was changed as follows, it would "just work", even when each 
iteration took a few seconds (6 seconds per iteration would give 1 minute for 
each execution of the timeit method):

class Timer:
     def __init__(self, stmt="pass", setup="pass", timer=default_timer):
         """Constructor.  See class doc string."""
         # ... existing constructor
         self.default_number = None

     def timeit(self, number=None): # Note changed default
         if number is None:
             if self.default_number is None:
                 self.default_number = self.calibrate()
             number = self.default_number
         # ... existing timeit method

     def calibrate(self): # New method, taken from script code at end of module
         # find smallest power of 10 >= 10 so that 0.2 seconds <= total time
         # capped at 10 billion (10e10) iterations, as reaching that limit
         # implies fewer than 20 picoseconds (2e-11) per iteration (i.e. Fast!)
         for i in range(1, 10):
             number = 10**i
             x = self.timeit(number)
             if x >= 0.2:
                 break
         self.default_number = number


>> Secondly, there should be a way to time a callable
>> directly.  That is, without finding the string name of
>> the callable and using a string "import X" statement. 
>> These contortions violate rules #1 and #3 of the Zen
>> of Python.
>>
> Presumably for benchmarking purposes the function call overhead would be 
> present for all compaered techniques. Do you mean rules #0 and #2?

Timing an existing function really is a pain - timeit expects source code it 
can plug into a code template, so timing an existing function from an 
interactive session isn't as easy as it could be (you have to figure out a way 
to give strings to Timer that it can plug into its string template, rather 
than just giving it the callable directly).

A subclass would deal with that quite handily:

class CallTimer(Timer):
     # Use lambda or functional.partial to pass arguments to the callable
     def __init__(self, callable, timer=default_timer):
         def inner(_it, _timer):
             _t0 = _timer()
             for _i in _it:
                 callable()
             _t1 = _timer()
             return _t1 - _t0
         self.timer = timer
         self.inner = inner
         self.src = None

     # Using a real function, so leave linecache alone when printing exceptions
     def print_exc(self, file=None):
         import traceback
         traceback.print_exc(file=file)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org