memory control in Python

[Ping Liu]

Dear All,

I am working on an optimization problem, where we are trying to minimize
some indicators like energy usage, energy cost, CO2 emission. In this
problem, we have a bunch of energy conversion technologies for electricity
and thermal purpose, such as heat pump, boiler, chiller, etc.. We are
trying to model it with a one year time period. the current time step is
one hour.

We have preselected the  candidate technologies to exclude those we don't
want to study so that the problem size could be reduced with a limited
candidate technologies. In the current case study, we only analyze the
electric chiller and heat pump to supply the cooling load, while power grid
will supply the electricity for all electric loads. There are binary
variables regarding installation decisions of technologies and continuous
variables related to installation capacity and hourly operational
decisions.

For small cases, Python works well. But if we consider longer time period.
then it would fail due to the memory usage issues. We have tested several
case studies to check the memory use for different time period, including
1) 2 hours in one day, 2) 24 hours in one day, 3) 20 days with 24 hours
each day, as well as 4) 30 days with 24 hours each day. The first 3 cases
are feasible while the last case gives out the memory error.

When we are testing the forth case, the memory error comes out while
creating the inequality constraints. The problem size is 1) Aeq: 12 * 26,
Aineq: 30 * 26; 2) Aeq: 144*268, Aineq:316*268; 3) Aeq: 2880*5284, Aineq:
6244*5284; 4) Aeq: 4320 * 7924, Aineq is unknown due to the memory error.

The solver is CPLEX (academic). It turns out that the solver is taking a
lot of memory as you can see in the memory test report. for the first three
cases, different memory usage is observed, and it grows up dramatically
with the increase of the time period. 1) solver memory usage: 25.6 MB, 2)
19.5 MB; 3) solver memory usage: 830.0742 MB.

Based on my observations, I have some of the following questions regarding
1) In order to create the optimization problem (Aeq, Aineq), how can we
reduce the memory usage in python programming?  2) how can I reduce the
memory usage of different solvers in Python? Why would the memory decrease
for the CPLEX solver within the 24-hours-in-one-day case compared with the
case 1?

Thanks in advance,
Ping
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150814/cc830eb3/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: solver memory comparison.xlsx
Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Size: 8549 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150814/cc830eb3/attachment.xlsx>