execution time

[Steven D'Aprano]

On Sun, 14 Dec 2008 18:35:24 +0100, Andreas Kostyrka wrote:

> On Sun, Dec 14, 2008 at 05:03:38PM +0100, David Hláčik wrote:
>> Hi guys,
>> 
>> #! /usr/bin/python
>> 
>> import random
>> import bucket2
>> 
>> data = [ random.randint(1,25) for i in range(5)] print "random data :
>> %s" % data
>> print "result: %s" %bucket2.sort(data)
>> 
>> How to write a test script which will outputs execution time for
>> bucket2.sort(data) ?
> 
> Well:
> 
> import time
> 
> starttime = time.time()
> 
> ....
> 
> endtime = time.time()
> print "Whatever .... does, it took %5.3fs to do." % (endtime -
> starttime)


Is subject to lots of timing errors for small code snippets. Sorting five 
numbers is (probably) going to be very quick, so the random timing errors 
will probably be larger than the time it actually takes to sort!


Just for reference, here's the size of errors from timing naively on my 
machine:

def timer(alist):
    from time import time
    t = time()
    alist.sort()
    return time() - t

def make_list():
    from random import random
    return [random() for i in range(5)]

data = [timer(make_list()) for i in range(100)]
mean = sum(data)/len(data)
average_error = sum(abs(x-mean) for x in data)/len(data)

print "The average timing is %f seconds." % mean
print "The average error is %f seconds." % average_error
print "Percentage error: %f%%" % (average_error/mean*100)


And here is the output:

The average timing is 0.000050 seconds.
The average error is 0.000089 seconds.
Percentage error: 177.953157%


What this tells us is that the likely error in any one timing of a small 
code snippet using time.time() directly is so large that it is useless 
for anything other than the most crude measurements. Using time() on its 
own is a good example of "measure with micrometer, mark with chalk, cut 
with axe". This is precisely the problem the timeit module is written to 
solve.



> Alternativly take a look at the timeit standard module.

I'd reverse that recommendation: start with the timeit module first, and 
if and only if it is unsuitable, consider writing your own solution.


-- 
Steven