[Numpy-discussion] N dimensional dichotomy optimization

[Sebastian Walter]

On Tue, Nov 23, 2010 at 2:50 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> On Tue, Nov 23, 2010 at 02:47:10PM +0100, Sebastian Walter wrote:
>> Well, I don't know what the best method is to solve your problem, so
>> take the following with a grain of salt:
>> Wouldn't it be better to change the model than modifying the
>> optimization algorithm?
>
> In this case, that's not possible. You can think of this parameter as the
> number of components in a PCA (it's actually a more complex dictionnary
> learning framework), so it's a parameter that is discrete, and I can't do
> anything about it :).

In optimum experimental design one encounters MINLPs where integers
define the number of rows of a matrix.
At first glance it looks as if a relaxation is simply not possible:
either there are additional rows or not.
But with some technical transformations it is possible to reformulate
the problem into a form that allows the relaxation of the integer
constraint in a natural way.

Maybe this is also possible in your case?
Otherwise, well, let me know if you find a working solution ;)


>
>> It sounds as if the resulting objective function is piecewise
>> constant.
>
>> AFAIK most optimization algorithms for continuous problems require at
>> least Lipschitz continuous functions to work ''acceptable well''. Not
>> sure if this is also true for Nelder-Mead.
>
> Yes correct. We do have a problem.
>
> I have a Nelder-Mead that seems to be working quite well on a few toy
> problems.
>
> Gaël
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>