[SciPy-Dev] SciPy-Dev Digest, Vol 203, Issue 6

Fri Sep 25 21:52:31 EDT 2020

 Hello,
My colleague and I discussed this and now agree that this should be handled in the function being optimized, not in the scipy differential_evolution code. We do feel, however, that it would be good to add documentation for handling such cases, especially when the optimum winds up being at a boundary where np.inf occurs. We would be happy to submit draft documentation to add to https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.differential_evolution.html if that is of interest.
Let me explain some of the details of issues we ran across. Interestingly, in a side conversation we learned that a similar thing came up in issue #7799/7819.
In our case, the function is actually a solution of a set of coupled boundary value ode's. For some values of the boundary condition, there are no feasible solutions, but it would be difficult in many cases to explicitly map out boundaries with all infeasible regions and add those as bounds.  In our initial attempts to use the differential evolution solver, the scipy code failed when our solver of the ode's gave warnings due to infeasibilities. We did learn to turn those warnings into errors and catch them with try-except blocks, and then return np.inf if the exceptions happened. Even that might be useful for some people to see described in documentation :) This is a case of infeasible solutions. But another problem we encountered is that sometimes the optimum then winds up being very near or actually at the boundary with an infeasible region. In such a case, the differential evolution code does not converge or behave well because each time it picks a new population, some points are in the infeasible region (since the "answer" is at the border), and therefore there are np.inf values that continue to get returned and thereby interfere with the concept of shrinking the population towards the answer.  We handled this by using a callback function that gave early termination if the "best value" is not changing. I feel that would also be useful to document. Initially we were proposing to add to the scipy code itself to recognize this is happening, and do something about it internally in scipy. But we now agree that having this different convergence criterion is better handled with a callback function, as it is specific to the problem being encountered and is consistent with the availability of the callback function.
A similar problem also occurred in issue #7799/7819 for basinhopping optimization. In that case, there was not a boundary where solutions for the function became infeasible, but rather there was an explicit constraint that wound up causing the solution to be exactly at the constraint boundary. The effect was the same as above because the solution was no longer a local minimum.  Returning np.inf when appropriate and adding a callback function that provides early termination when the objective function is not changing works for that case as well.
Would it be of interest to have this added to the documention?
Best regards,Frank 

    On Wednesday, September 16, 2020, 02:59:09 PM PDT, scipy-dev-request at python.org <scipy-dev-request at python.org> wrote:  

 Send SciPy-Dev mailing list submissions to
    scipy-dev at python.org

To subscribe or unsubscribe via the World Wide Web, visit
    https://mail.python.org/mailman/listinfo/scipy-dev
or, via email, send a message with subject or body 'help' to
    scipy-dev-request at python.org

You can reach the person managing the list at
    scipy-dev-owner at python.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of SciPy-Dev digest..."
Today's Topics:

  1. feature proposal for scipy.optimize.differential_evolution
      (frank.home at sbcglobal.net)
  2. Re: feature proposal for
      scipy.optimize.differential_evolution (Matt Haberland)
  3. Re: feature proposal for
      scipy.optimize.differential_evolution (Andrew Nelson)

Hello,
In using scipy.optimize.differential_evolution, we found that it is useful to modify the behavior for problems where solutions of the equations being optimized are not feasible in some of the space being explored. Rather than having the code throw exceptions and then abort, we use the approach of returning np.inf for the objective function in infeasible regions. We are interested in turning this approach into a feature of scipy.optimize.differential_evolution and submitting a pull request, and would like to know if people think this would be a useful contribution. 
Our approach would be to add a boolean parameter to scipy.optimize.differential_evolution for selecting whether to handle lack of ability to find a solution in the above manner. The default for the parameter would be to not do this, i.e. run scipy.optimize.differential_evolution as it now works.
Please let us know what you think, and if you would be interested in reviewing code, please let us know that as well!
Thank you,Frank Torres

  Sounds useful to me to have that option. Do you know of other software with a similar option?
On Wed, Sep 16, 2020 at 10:09 AM frank.home at sbcglobal.net <frank.home at sbcglobal.net> wrote:

Hello,
In using scipy.optimize.differential_evolution, we found that it is useful to modify the behavior for problems where solutions of the equations being optimized are not feasible in some of the space being explored. Rather than having the code throw exceptions and then abort, we use the approach of returning np.inf for the objective function in infeasible regions. We are interested in turning this approach into a feature of scipy.optimize.differential_evolution and submitting a pull request, and would like to know if people think this would be a useful contribution. 
Our approach would be to add a boolean parameter to scipy.optimize.differential_evolution for selecting whether to handle lack of ability to find a solution in the above manner. The default for the parameter would be to not do this, i.e. run scipy.optimize.differential_evolution as it now works.
Please let us know what you think, and if you would be interested in reviewing code, please let us know that as well!
Thank you,Frank Torres

  _______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org
https://mail.python.org/mailman/listinfo/scipy-dev

-- 
Matt HaberlandAssistant ProfessorBioResource and Agricultural Engineering08A-3K, Cal Poly
On Thu, 17 Sep 2020 at 03:09, frank.home at sbcglobal.net <frank.home at sbcglobal.net> wrote:

In using scipy.optimize.differential_evolution, we found that it is useful to modify the behavior for problems where solutions of the equations being optimized are not feasible in some of the space being explored. Rather than having the code throw exceptions and then abort, we use the approach of returning np.inf for the objective function in infeasible regions. We are interested in turning this approach into a feature of scipy.optimize.differential_evolution and submitting a pull request, and would like to know if people think this would be a useful contribution. 

Our approach would be to add a boolean parameter to scipy.optimize.differential_evolution for selecting whether to handle lack of ability to find a solution in the above manner. The default for the parameter would be to not do this, i.e. run scipy.optimize.differential_evolution as it now works.

As the code currently exists the user's objective `func` is allowed to return `np.inf` if it finds itself within a region that is infeasible, differential_evolution doesn't throw exceptions or abort if `func` does that. One can also mark regions as infeasible with constraint functions.
So it's not clear to me what feature is missing. Do you have a demonstration?

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org
https://mail.python.org/mailman/listinfo/scipy-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200926/f087eaf9/attachment-0001.html>