Speeding compile()/eval() (was : Slowness in compile()/eval())
Pedro Rodriguez
pedro_rodriguez at club-internet.fr
Fri Jan 18 09:38:11 EST 2002
Thanks to Alex Martelli hints, I have a better understanding
of the way Python deals with variable lookups. Here after
is the idiom.
Comments and cleanup will be welcomed.
TIA.
Idiom :
If you need to compile an expression in order to do use it
many times in a computation, instead of just :
code = compile("...expr...", '<string>', 'eval')
...
loop...
val = eval(code)
...
wrapping your expression (and probably part of the computation)
in a function can improve the performance :
code = compile("""
def f(...some variables used by the expression):
return %s
""" % "...expr...", '<string>', 'exec')
...
loop...
val = f(...variables...)
...
the gain expected is due to the fact that the lookup for
local variable is faster than the one used in the compile
code that goes through local and global lookup.
The main caution has to be put on identifying the variables
to be put in the function args.
----------------------------------------------------------------------
The example exihibit some speed acceleration :
t1 : with compile(), eval()
t2 : with compile(func), func(), 50% speed up
t3 : with compile(algo), algo(), 75% speed up
Results for RH7.2, PIII, 800Mhz
Python 1.5.2
t1 : 14.759
t2 : 6.832
t3 : 3.153
Python 2.1.1
t1 : 12.156
t2 : 5.183
t3 : 2.679
Python 2.2
t1 : 12.806
t2 : 5.987
t3 : 2.992
----------------------------------------------------------------------
example.py
----------------------------------------------------------------------
SIZE = 1000000
RSIZE = range(SIZE)
def t1():
a = [ 0 ] * SIZE
b = [ 1 ] * SIZE
d = [ None ] * SIZE
exprStr = "a[i]*a[i]+b[i]*b[i]+2.0*a[i]*b[i]"
code = compile(exprStr, '<string>', "eval")
for i in RSIZE:
d[i] = eval(code)
def t2():
a = [ 0 ] * SIZE
b = [ 1 ] * SIZE
d = [ None ] * SIZE
exprStr = "a[i]*a[i]+b[i]*b[i]+2.0*a[i]*b[i]"
code = compile("""
def f(a,b,i):
return %s
""" % exprStr, '<string>', "exec")
exec code
for i in RSIZE:
d[i] = f(a,b,i)
def t3():
a = [ 0 ] * SIZE
b = [ 1 ] * SIZE
d = [ None ] * SIZE
exprStr = "d[i] = a[i]+b[i]/2.0"
code = compile("""
def f(a,b,d):
for i in RSIZE:
d[i] = %s
""" % exprStr, '<string>', "exec")
exec code
f(a,b,d)
def bench(t):
import time
t0 = time.time()
t()
t1 = time.time()
print "%5s : %6.3f" % (t.func_name, t1-t0)
bench(t1)
bench(t2)
bench(t3)
----------------------------------------------------------------------
--
Pedro
More information about the Python-list
mailing list