[SciPy-User] Artificial Neural Network thoughts

[Gerrit Holl]

Hi,

Summary of e-mail: "There are so many ANN packages and none seem very
good; which one should I choose?"

I am in the process of a transition from using primarily Matlab™ to
using Python and additional packages. I've been getting increasingly
annoyed with Matlab™ for a long time for reasons I probably don't need
to explain here. My colleagues often accuse OSS of being too instable;
not in the sense that it crashes, but that API's change too often and
code just breaks upon increasing versions. Indeed, to get a several
years old Python package written by a former co-worker working, I
spend some hours digging through the code, because scipy now requires
explicit imports (see
http://projects.scipy.org/scipy/changeset/5206/trunk/scipy/__init__.py
), and this package relied on "from scipy import *'. Not nice, I don't
see a good reason why this should break. But as Matlab™ in its latest
upgrade from 2010A to 2010B started to do the same (changing error
identifiers, changing behaviour in neural network toolbox, both
undocumented), I decided to go for Python. Of course, Python is not
enough; for my work, I need at least numpy, scipy, matplotlib,
ScientificPython, pytables (or another HDF5 interface), and a neural
network toolkit.

Many dependencies for my code, but alas, it works. I found more or
less actively maintained packages that do what I need for all but the
last toolkit. I need to train a regression, with neural networks or
maybe other machine learning systems (support vector machines,
bayesian monte carlo integration, ...). And frankly, I'm a bit worried
about what's "out there". I like OSS, but here it appears considerably
less strong than the propriety ANN package from Mathworks™,
particularly when it comes to documentation. I am looking for advice.
Is anyone using ANNs in production code, if so, what are the
experiences? I have found:

- Fast Artificial Neural Network Library (FANN), written in C but with
Python bindings. Looks good, but appears to lack active maintenance.
Version 2.1.0beta was published 2007-02-01 and the CVS repository
appears unchanged for years. That doesn't sound good. Do I really want
to rely on that?
- bpnn.py, a 171 LOC lone pure Python file implementing
back-propagation NN that "could use numpy to speed this up". Looks
really simple and probably quite good to understand the concept, but
is it any good for production code? Looks like I would need to do a
lot of coding to make it fast (at least numpy-based) and tweak the
settings. A Numeric-based implementation exists at
http://www.dacya.ucm.es/jam/download.htm. It would probably take some
time porting it to numpy.
- ffnet, last (beta...) version published 2009-10-27 and last activity
in svn tree 9 months ago. Seems allright, but is it stable? Is it
maintained? Can I use it?
- Monte Python appears to do neural networks. No idea how good it is.
Last published version in June this year, verison 0.2.0, "introduces
incompatibilities with previous versions" and was published more than
2 years after the version before.
- pyann, http://sourceforge.net/projects/pyann/ "version 0.1.0"
published in 2007 and last svn activity 4 months ago. Another one that
seems really fragile.

There are others. That is in fact a disadvantage. There are >5
packages and as far as I can see none sticks out. Do I really have the
time to try them all out? Different needs, different qualities. But
active maintenance is something anyone will be interested in, I
suppose. Hence my question:

What experiences do people have in using Artificial Neural Networks in
Python+friends? What packages are suitable for production code and can
be more or less relied on without having to understand all the
details? Is any package much more widely used than the others?

Thoughts are welcome!

regards,
Gerrit.

-- 
Gerrit Holl
PhD student at Department of Space Science, Luleå University of
Technology, Kiruna, Sweden
http://www.sat.ltu.se/members/gerrit/