A performance issue when using default value

[Steven D'Aprano]

On Sun, 31 Jan 2010 20:58:50 -0800, keakon wrote:

> I've found strange performance issue when using default value, the test
> code is list below:
> 
> from timeit import Timer
> 
> def f(x):
>   y = x
>   y.append(1)
>   return y
> 
> def g(x=[]):
>   y = []
>   y.append(1)
>   return y
> 
> def h(x=[]):
>   y = x
>   y.append(1)
>   return y
> 
> def f2(x):
>   y = x
>   y.append(1)
>   return y + []
> 
> def g2(x=[]):
>   y = []
>   y.append(1)
>   return y + []
> 
> def h2(x=[]):
>   y = x
>   y.append(1)
>   return y + []
> 
> TIMES = 10000
> print Timer('f([])','from __main__ import f, g, h').timeit(TIMES) print
> Timer('g()','from __main__ import f, g, h').timeit(TIMES) print
> Timer('h([])','from __main__ import f, g, h').timeit(TIMES) print
> Timer('h()','from __main__ import f, g, h').timeit(TIMES) print
> Timer('f2([])','from __main__ import f2, g2, h2').timeit(TIMES) print
> Timer('g2()','from __main__ import f2, g2, h2').timeit(TIMES) print
> Timer('h2([])','from __main__ import f2, g2, h2').timeit(TIMES) print
> Timer('h2()','from __main__ import f2, g2, h2').timeit(TIMES)
> 
> 
> I tested it with Python 2.5.4, 2.6.4 and 3.1.1 on Windows XP, and get
> almost the same result:
> 0.00449247041174
> 0.00439608944712
> 0.00455867994396
> 0.00327471787615
> 0.00791581052899
> 0.00684919452053
> 0.00734311204357
> 0.30974942346


You've only run those results once, haven't you?

That is not trustworthy. In a multi-tasking operating system, including 
Windows, some other process might have run for some milliseconds, 
throwing the results off.

More importantly, if you had run it multiple times, you'd see this:


>>> for i in range(7):
...     Timer('h2()','from __main__ import f2, g2, h2').timeit(TIMES)
...
0.0190200805664
0.315117120743
0.941411972046
1.56618499756
2.18553495407
2.79832291603
3.44865894318


h2 (and probably h, but I haven't tested it) get slower and slower and 
slower every time you run it.

Why? I'm pretty sure this is in the FAQs, but default values are created 
once, not every time the function is called. So you create a function 
with the default value of []. Call the function, and that list gets 1 
appended to it, so the default value is now [1]. Call the function again, 
and it becomes [1, 1]. And so on, and so on, and so on.

The other functions create a new empty list each time the function is 
called, so you're comparing the time it takes to grow an ever-bigger list 
with the time it takes to create a tiny list.




-- 
Steven