[SciPy-User] scipy.sparse.linalg.cg statistics?

[josef.pktd at gmail.com]

On Fri, Oct 8, 2010 at 12:30 PM, Pauli Virtanen <pav at iki.fi> wrote:
> Fri, 08 Oct 2010 17:42:10 +0200, Nico Schlömer wrote:
> [clip]
>> There's a major drawback, though, which may not be easily fixed: To
>> calculate the residual, one has to compute an extra matrix-vector
>> multiplication *per step*, which effectively increases the runtime of
>> the CG algorithm by a factor of two (!). Ideally -- and classically --
>> the residual is retrieved directly from the CG method itself. Now
>> obviously, the callback method does not provide for residual vector
>> input, but I'm not quite sure what's the reason for that.
>
> That can quite likely be easily fixed by making a small change to Scipy.
> An interesting point here is that the `gmres` routine on the other hand
> only passes the residual norm to the callback, so there would be also
> some other things to harmonize here.
>
> Is there some other information that would be useful in the callback? I
> can think of: residual vector, residual 2-norm, and current iterate --
> what else?
>
> If you have time, please file a bug report,
>
>    http://projects.scipy.org/scipy/newticket
>
> so that we won't forget this issue.

A related idea:

Is it possible or is there a trick to tell an optimizer from a
callback function to stop?

I never used callback function, so I don't know what's possible.

I have two usecases in mind:
Stata for example switches optimizers during estimation of a
statistical model. It's possible to limit the number of iterations,
and restart with a different optimizer but it might be useful to
switch depending on the state of the optimization.
fmin_bfgs doesn't look very robust, quite often it goes of into
neverland and I can wait quite some time until I can restart with a
different optimizer. It would be useful in this case if we can inspect
what's going on and kill it automatically and switch to something more
robust.

Josef




>
> --
> Pauli Virtanen
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>