[Numpy-discussion] Improvement of performance

[Guilherme P. de Freitas]

I forgot to mention one thing: if you are doing optimization, a good
solution is a modeling package like AMPL (or GAMS or AIMMS, but I only
know AMPL, so I will restrict my attention to it). AMPL has a natural
modeling language and provides you with automatic differentiation.
It's not free, but there are trial licenses (60 days) and student a
student edition (unlimited time, maximum of 300 variables). It is
hooked to many great solvers, including KNITRO (commercial) and IPOPT
(free), both for nonlinear programs.

http://www.ampl.com

And as this is a Python list, there is NLPy. I never tried it, but it
seems to allow you to read model files written in AMPL, use AMPL's
automatic differentiation capabilities,  and still roll your own
optimization algorithm, all in Python. It looks like it requires some
other packages aside from Python, NumPy and AMPL, like a sparse linear
solver and some other things. Worth taking a look.

http://nlpy.sourceforge.net/how.html

Best,

Guilherme

On Tue, May 4, 2010 at 5:23 PM, Guilherme P. de Freitas
<guilherme at gpfreitas.com> wrote:
> On Tue, May 4, 2010 at 2:57 PM, Sebastian Walter
> <sebastian.walter at gmail.com> wrote:
>> playing devil's advocate I'd say use Algorithmic Differentiation
>> instead of finite differences ;)
>> that would probably speed things up quite a lot.
>
> I would suggest that too, but aside from FuncDesigner[0] (reference in
> the end), I couldn't find any Automatic Differentiation tool that was
> easy to install for Python.
>
> To stay with simple solutions, I think that the "complex step"
> approximation gives you a very good compromise between ease of use,
> performance and accuracy.  Here is an implementation (and you can take
> ideas from it to get rid of your "for" loops in your original code)
>
> import numpy as np
>
> def complex_step_grad(f, x, h=1.0e-20):
>    dim = np.size(x)
>    increments = np.identity(dim) * 1j * h
>    partials = [f(x+ih).imag / h for ih in increments]
>    return np.array(partials)
>
> **Warning**: you must convert your original real-valued function f:
> R^n -> R to the corresponding complex function f: C^n -> C. Use
> functions from the 'cmath' module.
>
> I strongly suggest that you take a look at the AutoDiff website[1] and
> at some references about the complex step [2][3] (or just Google
> "complex step" and "differentiation", both on normal Google and Google
> Scholar).
>
> [0] http://openopt.org/FuncDesigner
> [1] http://www.autodiff.org/
> [2] http://doi.acm.org/10.1145/838250.838251
> [3] http://mdolab.utias.utoronto.ca/resources/complex-step
>
>
> --
> Guilherme P. de Freitas
> http://www.gpfreitas.com
>



-- 
Guilherme P. de Freitas
http://www.gpfreitas.com