[Python-ideas] Preventing out of memory conditions

[Gregory P. Smith]

On Mon, Dec 31, 2012 at 7:28 PM, Random832 <random832 at fastmail.us> wrote:

> On 12/31/2012 7:11 PM, Ben Hoyt wrote:
>
>> Interesting idea, though I don't think it's something that should be a
>> Python language extension. For instance, iOS (typically more
>> resource-constrained) sends the application signals when system memory is
>> getting low so it can free stuff -- this is done at the OS level. And I
>> think that's the right place, because this will almost certainly be setup-
>> and system-dependent. For instance, it would depend hugely on whether
>> there's a virtual memory manager, and how it's configured.
>>
>> I'd say your best bet is to write a little library that does the
>> appropriate thing for your needs (your system or setup). Say starts a
>> thread that checks the system's free memory every so often and sends your
>> application a signal/callback saying "we're getting low, free some of your
>> caches". It could even send a "level flag" to your callback saying "fairly
>> low", "very low", or "critically low" -- I think iOS does this.
>>
>
> I'm concerned that a program that does this will end up as the loser in
> this scenario:
>
> http://blogs.msdn.com/b/**oldnewthing/archive/2012/01/**18/10257834.aspx<http://blogs.msdn.com/b/oldnewthing/archive/2012/01/18/10257834.aspx>
>
> (tl;dr, two programs each having a different idea of how much free memory
> the system should have results in an "unfair" total allocation of memory)
>
> I think it's possibly important to avoid using the system's free memory as
> an input to any such system.
>
>
indeed. only look at your own process's consumption vs. some numerical
limit you've chosen for yourself.  this also means you can adjust your own
limit up or down at runtime if desired.  (JVM's tend to force you to work
this way)

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121231/539af667/attachment.html>