[SciPy-Dev] Returning the (approximate) inverse hessian from L-BFGS-B as a LinearOperator?

[Pauli Virtanen]

Hi,

18.02.2015, 03:18, Robert McGibbon kirjoitti:
[clip]
> In the L-BFGS-B minimizer, a low-memory approximation to the inverse
> hessian is
> used internally to determine the descent direction. Once the optimization
> terminates,
> none of this information is returned to the client though. I think it would
> be a nice to
> return a callable for the inverse hessian - vector product as a
> LinearOperator.

Depending on what the implicit storage format in lbfgsb is, the
Python-side may already be implemented in
scipy.optimize.nonlin.LowRankMatrix

The maintenance problem here is that extracting the information requires
mucking with undocumented implementation details of the algorithm, which
may change the author releases yet another new version.

However, the software is fairly mature, so with proper tests adding this
could be possible.

A completely separate question then is whether the approximation is
useful for your purpose, and whether you could do better just by
constructing the BFGS secant approximation yourself by recording the
gradients evaluated during your optimization process.