Testing for performance regressions

[Paddy]

In an extended case when you try and capture how a function works over a range of inputs, you might want to not assume some relationship between input size and time, as this mnight limit your ability to change algorithms and still have acceptable performance. 

I.e. instead of this:

input_range = (MIN, AVERAGE, MAX)
for i in inpute_range:
..baseline = Timer(simple_func(i)).timeit()
..time_taken = Timer(my_func(i)).timeit()
..assert time_taken <= simple_relation(i) * baseline

It might be better to do this:

input_range = (MIN_R, AVERAGE_R, MAX_R)
time_ranges = (MIN_T, AVERAGE_T, MAX_T)
for i,t in zip(inpute_range, time_ranges):
..baseline = Timer(simple_func(i)).timeit()
..time_taken = Timer(my_func(i)).timeit()
..assert time_taken <= t * baseline

This comes from electronic circuit design where designs must be proven to work over a range for different values for example voltage ranges and temperature ranges. 

The action of the function being timed might not be simple, for example if my_func swapped algorithms depending on its input to favour the average case.