[SciPy-dev] Some clarification....

[eric]

Hey Pat,

> for speed.  I'll probably put in a more general Python model
> (in my copious spare time!) because loops are so slow that
>
> for i in range(n):
>      < anything>
>
> will run MUCH faster as a C loop.  You can get an idea of the
> over head by trying the difference between
>
> for i in xrange(n): f(i)
>
> vs
>
> map(f,xrange(n))
>
> and see the speed difference from simply doing the loop inside
> C vs inside Python.

I think this used to be a lot worse than it is now.  The following simple
tests only shows modest improvement:

>>> def bob(a): return float(a)**2+a/a*10
...
>>> def q():
...     result = []
...     for i in range(1,100000):
...       result.append(bob(i))
...
>>> t1 = time.time();q(); print time.time() - t1
0.81200003624
>>> def r():
...    map(bob,range(1,100000))
...
>>> t1 = time.time();r(); print time.time() - t1
0.600999951363
>>> def r():
...    map(bob,xrange(1,100000))
...
>>> t1 = time.time();r(); print time.time() - t1
0.600999951363

33% improvement is decent, but there are definitely bigger fish to fry.

> A story will illustrate....
>
> Back in the old days, when Crays were the workhorses at my
> Lab, Cray provided a hand tuned FFT that carefully tried to
> minimize bank conflicts and enhance vectorization and parallelism.
> My boss wrote one that was 10% better in about six weeks using
> a special purpose high level language (Sisal if you're interested)
> that did wonders.

Man, I can hardly imagine spending 6 weeks for 10%.  He should get out
more... :-)

see ya,
eric