[Numpy-discussion] Numpy and PEP 343

[eric jones]

Travis Oliphant wrote:

>
> Weave can still help with the "auto-compilation" of the specific 
> library for your type.  Ultimately such code will be faster than NumPy 
> can every be.

Yes.  weave.blitz() can be used to do the equivalent of this lazy 
evaluation for you in many cases without much effort.  For example:

import weave
from scipy import arange

a = arange(1e7)
b = arange(1e7)
c=2.0*a+3.0*b

# or with weave
weave.blitz("c=2.0*a+3.0*b")

As Paul D. mentioned.what Tim outlined is essentially template 
expressions in C++.  blitz++ (http://www.oonumerics.org/blitz/) is a C++ 
template expressions library for array operations, and weave.blitz 
translates a Numeric expression into C++ blitz code.  For the example 
above on large arrays, you get about a factor of 4 speed up on large 
arrays. (Notice, the first time you run the example it will be much 
slower because of compile time.  Use timings from subsequent runs.)

C:\temp>weave_time.py
Expression: c=2.0*a+3.0*b
Numeric: 0.678311899322
Weave: 0.162177084984
Speed-up: 4.18253848494

This isn't as good as you can do with hand coded C, but it isn't so bad 
for the effort involved...
I have wished for time to write a weave.f90("c=2.0*a+3.0*b") function 
because it is very feasible.  My guess from simple experimentation is 
that it would be about as fast as hand coded C for this sort of 
expression.  This might give us another factor of two or three in 
execution speed and the compile times would come down from tens of 
seconds to tenths of seconds.

Incidentally, the weave calling overhead is large enough to limit its 
benefit on small arrays.  Pat Miller pointed out some ways to get rid of 
that overhead, and I even wrote some experimental fixes to weave that 
helped out a lot.  Alas, they never were completed fully.  Revisiting 
these would make weave.blitz useful for small arrays as well.  Fixing 
these is probably more work than wrting blitz.f90()

All this to say, I think weave basically accomplishes what Tim wants 
with a different mechanism (letting C++ compilers do the optimization 
instead of writing this optimization at the python level).  It does 
require a compiler on client machines in its current form (even that can 
be fixed...), but I think it might prove faster than re-implementing a 
numeric expression compiler at the python level (though that sounds fun 
as well).

see ya,
eric

###############################
#weave_time.py
###############################
import timeit

array_size = 1e7
iterations = 10

setup = """\
import weave
from scipy import arange
a=arange(%f)
b=arange(%f)
c=arange(%f) # needed by weave test
""" % (array_size, array_size, array_size)

expr = "c=2.0*a+3.0*b"
print "Expression:", expr

numeric_timer = timeit.Timer(expr, setup)
numeric_time = numeric_timer.timeit(number=iterations)
print "Numeric:", numeric_time/iterations

weave_timer = timeit.Timer('weave.blitz("%s")' % expr, setup)
weave_timer = timeit.Timer('weave.blitz("%s")' % expr, setup)
weave_time = weave_timer.timeit(number=iterations)
print "Weave:", weave_time/iterations

print "Speed-up:", numeric_time/weave_time


> -Travis
>
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting 
> language
> that extends applications into web and mobile media. Attend the live 
> webcast
> and join the prime developer group breaking into this new coding 
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion