[SciPy-User] scipy.optimize named argument inconsistency

[josef.pktd at gmail.com]

On Tue, Sep 6, 2011 at 10:33 AM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
> On Tue, Sep 6, 2011 at 8:16 AM, <josef.pktd at gmail.com> wrote:
>>
>> On Mon, Sep 5, 2011 at 2:36 PM, Matthew Newville
>> <matt.newville at gmail.com> wrote:
>> >
>> >
>> > On Monday, September 5, 2011 8:23:41 AM UTC-5, joep wrote:
>> >>
>> >> On Sun, Sep 4, 2011 at 8:44 PM, Matthew Newville
>> >> <matt.n... at gmail.com> wrote:
>> >> > Hi,
>> >> >
>> >> > On Friday, September 2, 2011 1:31:46 PM UTC-5, Denis Laxalde wrote:
>> >> >>
>> >> >> Hi,
>> >> >>
>> >> >> (I'm resurrecting an old post.)
>> >> >>
>> >> >> On Thu, 27 Jan 2011 18:54:39 +0800, Ralf Gommers wrote:
>> >> >> > On Wed, Jan 26, 2011 at 12:41 AM, Joon Ro <joo... at gmail.com>
>> >> >> > wrote:
>> >> >> > > I just found that for some functions such as fmin_bfgs, the
>> >> >> > > argument
>> >> >> > > name
>> >> >> > > for the objective function to be minimized is f, and for others
>> >> >> > > such
>> >> >> > > as
>> >> >> > > fmin, it is func.
>> >> >> > > I was wondering if this was intended, because I think it would
>> >> >> > > be
>> >> >> > > better to
>> >> >> > > have consistent argument names across those functions.
>> >> >> > >
>> >> >> >
>> >> >> > It's unlikely that that was intentional. A patch would be welcome.
>> >> >> > "func"
>> >> >> > looks better to me than "f" or "F".
>> >> >>
>> >> >> There are still several inconsistencies in input or output of
>> >> >> functions
>> >> >> in the optimize package. For instance, for input parameters the
>> >> >> Jacobian
>> >> >> is sometimes name 'fprime' or 'Dfun', tolerances can be 'xtol' or
>> >> >> 'x_tol', etc. Outputs might be returned in a different order, e.g.,
>> >> >> fsolve returns 'x, infodict, ier, mesg' whereas leastsq returns 'x,
>> >> >> cov_x, infodict, mesg, ier'. Some functions make use of the infodict
>> >> >> output whereas some return the same data individually. etc.
>> >> >>
>> >> >> If you still believe (as I do) that consistency of optimize
>> >> >> functions should be improved, I can work on it. Let me know
>> >> >
>> >> > Also +1.
>> >> >
>> >> > I would add that the call signatures and return values for the
>> >> > user-supplied
>> >> > function to minimize should be made consistent too.  Currently, some
>> >> > functions (leastsq) requires the return value to be an array, while
>> >> > others
>> >> > (anneal and fmin_l_bfgs_b) require a scalar (sum-of-squares of
>> >> > residual).
>> >> > That seems like a serious impediment to changing algorithms.
>> >>
>> >> I don't see how that would be possible, since it's a difference in
>> >> algorithm, leastsq needs the values for individual observations (to
>> >> calculate Jacobian), the other ones don't care and only maximize an
>> >> objective function that could have arbitrary accumulation.
>> >
>> > Well, I understand that this adds an implicit bias for least-squares,
>> > but
>> > if the algorithms receive an array from the user-function, taking
>> > (value*value).sum() might be preferred over raising an exception like
>> >     ValueError: setting an array element with a sequence
>>
>> I'd rather have an exception, but maybe one that is more explicit.
>> leastsq is efficient for leastsquares problems. If I switch to fmin or
>> similar, it's usually because I have a different objective function,
>> and I want to have a reminder that I need to tell what my objective
>> function is (cut and paste errors are pretty common).
>>
>> >
>> > which is apparently meant to be read as "change your function to return
>> > a
>> > scalar".  I don't see in the docs where it actually specifies what the
>> > user
>> > functions *should* return.
>> >
>> > I also agree with Charles' suggestion. Unraveling multi-dimensional
>> > arrays
>> > for leastsq (and others) would be convenient.
>>
>> I'm not quite sure what that means.
>>
>
> Assuming you are speaking of leastsq, unraveling multidimensional arrays
> means that the function can return, say, an (n,3) array. Currently leastsq
> only works with 1D arrays. The (n,3) case is convenient if you are fitting
> vector valued data where the residuals would most naturally also be vector
> valued.

What I don't understand is how you want the 3 arrays combined ?

one stacked least squares problem, adding all errors**2, e.g.

def errors(params,  y, x):
      err = y - f(x,params)
      return err.ravel()

or a weighted sum of the 3 errors,  return (err*weight).ravel()

or 3 separate optimization problems or multiobjective optimization?

We have similar problems with panel data or system of equations (case
1 and 2 above), but combining them to one model is part of the
definition of the objective function.

case 3, 3 separate optimization problems, can be conveniently done in
the linear case (linalg), but I don't see how leastsq could do it.

Josef


>
> <snip>
>
> Chuck
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>