[SciPy-User] Classification using neural networks

[Gael Varoquaux]

On Thu, Jul 26, 2012 at 06:26:55PM +0200, Sturla Molden wrote:
> I don't know why that is, SVMs are slow to train and use,

Hum. I guess it depends in which settings, but I disagree: well
implemented they are very fast in many settings.

> and I have yet to see that they out-perform an ANN.

They are just way easier to set up and tune. Seting up and tuning an ANN
can be a bit black magic.

> Perhaps it's because the latest edition of Numerical Receipes favour
> them over AANs,

In the machine learning community (which I know well) this is certainly
not an argument. These people are not afraid of implementing complex
algorithms.

> because SVMs supposedly are more transparent and easier to understand
> (I beg to differ).

I actually agree with NRs here.

> Multilayer ANNs trained with Levenberg-Marquardt and error
> backpropagation are among the most powerful non-linear regression and
> classification tools there are.

Granted for non-linearity, but most high-dimensional problem are well
solved with linear model.

> And by the way, SciPy already has an  LM-engine to train one
> (scipy.optimize.leastsq), all it takes is the code to compute the
> Jacobian by backpropagation.

Yeah, well, that's a bit of work, isn't it :)

Gael