switching to numpy and failing, a user story

[Fernando Perez]

greg.landrum at gmail.com wrote:

> After using numeric for almost ten years, I decided to attempt to
> switch a large codebase (python and C++) to using numpy. Here's are
> some comments about how that went.
> 
> - The code to automatically switch python stuff over just kind of
> works. But it was a 90% solution, I could do the rest by hand. Of
> course, the problem is that then the code is still using the old
> numeric API, so it's not a long term solution. Unfortunately, to switch
> to the numpy API one needs documentation, which is a problem; see
> below.

[ RNG issues, already addressed ] 

> - My extension modules just won't build because the new numpy stuff
> lives in a different location from where Numeric used to live. I
> probably could fix this, but without the documentation I can't figure
> out how to do that. I'd also need to figure out how to port my code to
> use the new numpy API instead of the compatibility layer, but I can't
> do that without docs either.

I have to call bull on this.  I happen to have the per-pay book, but I
recently converted a large codebase from Numeric to Numpy, and I actually
never had to open the book.  Travis has made sure that the
compatibility-related information is easy to find:

http://www.scipy.org/Converting_from_Numeric 

has most of what you need, and this page:

http://www.tramy.us/guidetoscipy.html

contains a link to the first two chapters FOR FREE.  A lot of what's needed
for a port is covered in there.

In addition, the C API is available here (as well as being obviously in the
source, which you have access to):

http://www.scipy.org/NumPyCapi

And the doc page has many more useful links:

http://www.scipy.org/Documentation

So yes, you have to buy the book.  Travis has sunk over a year of his time
into an absolutely herculean effort that provides working scientists with
tools that are better than anything money can buy (yes, I have access to
both Matlab and IDL, and you can't pay me enough to use them instead of
Python).  

And he has the gall to ask for some money for a 300 page book? How dare he,
when one can walk into any Barnes and Noble and just walk out of the store
with a cart full of books for free!

It's funny how I don't see anyone complaining about any of the Python books
sold here (or at any other publishing house):

http://www.oreilly.com/pub/topic/python

I recently was very happy to pay for the Twisted book, since I need Twisted
for a project and a well-organized book is a good complement to the
auto-generated documentation.


And finally, if you had porting problems, none were reported on any of the
numpy/scipy mailing lists (else you used a different name or email, since I
can't find traces of queries from you my gmail archive where I keep
everything posted on all the lists):

http://www.scipy.org/Mailing_Lists

Lots of people have been porting their codes recently, and inevitably some
have run into difficulties.  EVERY single time when they actually say
something on the list, Travis (and others as well) is very fast with
specific help on exactly how to solve the problems, or with bug fixes when
the problem happens to be a numpy bug discovered by the user.  But don't
take my word for it:

http://sourceforge.net/mailarchive/forum.php?thread_id=30703688&forum_id=4890

(and Francesc has found some really nasty things, given how pytables pushes
numpy far beyond where Numeric ever went, so this is not a light
compliment).

Look, I'm sure you had issues with your code, we all have.  But I want to
make sure that others don't take from your message the wrong impression
regarding numpy, its future, its quality as a scientific computing
platform, or Travis (I'd build the man a statue if I could :).  

The environment which is developing around Python for scientific computing
is nothing short of remarkable.  If you find issues in the process, ask for
help on the numpy/scipy lists and I'm sure you will receive some.  But
please refrain from spreading FUD.

Regards,

f