[Numpy-discussion] Array vectorization in numpy

[Carlos Becker]

Hi all. Thanks for the feedback.
My point is not to start a war on matlab/numpy. This comes out of my wish to
switch from Matlab to something more appealing.
I like numpy and python, being a proper language (not like matlab scripts,
whose syntax is patched and destroyed as new versions come up).
I am impressed at how complete numpy is, and with numexpr as well.

In my case, sometimes it is required to process 1k images or more, and 2x
speed improvement in this case means 2 hours of processing vs 4.
Someone would say 'switch to C/C++', but this is not my point. This thread
came up when comparing matlab and python to find whether performance is
somehow similar,
and even if numpy is a bit slower, I would not mind. However, 3x-4x is an
important difference.

I tried the operations with an output argument and didn't see much
difference. I have to try that further with other examples.
If I find something new I will let you know, I would be glad to switch to
numpy soon ;)

Cheers,
Carlos


On Wed, Jul 20, 2011 at 9:21 AM, Benjamin Root <ben.root at ou.edu> wrote:

> On Wednesday, July 20, 2011, Carlos Becker <carlosbecker at gmail.com> wrote:
> > Those are very interesting examples. I think that pre-allocation is very
> important, and something similar happens in Matlab if no pre-allocation is
> done: it takes 3-4x longer than with pre-allocation.The main difference is
> that Matlab is able to take into account a pre-allocated array/matrix,
> probably avoiding the creation of a temporary and writing the results
> directly in the pre-allocated array.
> >
> > I think this is essential to speed up numpy. Maybe numexpr could handle
> this in the future? Right now the general use of numexpr is result =
> numexpr.evaluate("whatever"), so the same problem seems to be there.
> >
> > With this I am not saying that numpy is not worth it, just that for many
> applications (specially with huge matrices/arrays), pre-allocation does make
> a huge difference, especially if we want to attract more people to using
> numpy.
>
> The ufuncs and many scipy functions take a "out" parameter where you
> can specify a pre-allocated array.  It can be a little awkward writing
> expressions that way, but the capability is there.
>
> But, ultimately, I think the main value with python and numpy is not
> it's speed, but rather the ease of use and how quickly one can develop
> working code with it.  If you want to squeeze every CPU resources, you
> could program in assembly, but good luck getting that linear solver
> done in time.
>
> Don't get me wrong, there is always room for improvements, and I would
> love to see numpy go even faster.  However, I doubt that converting
> matlab users would *require* speed to be the main selling point.  Ease
> of development and full-featured, high-quality standard and
> third-party libraries have always been the top selling points for me.
>
> Ben Root
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110720/84913fcf/attachment.html>