[Tutor] Question about the memory manager

[Albert-Jan Roskam]

> To: tutor at python.org
> From: __peter__ at web.de
> Date: Sun, 10 Jan 2016 18:29:06 +0100
> Subject: Re: [Tutor] Question about the memory manager
> 
> Albert-Jan Roskam wrote:
> 
> > Hi,
> > 
> > I just found a neat trick to free up an emergency stash of memory in a
> > funtion that overrides sys.excepthook. The rationale is that all
> > exceptions, including MemoryErrors will be logged. The code is below. My
> > question: is that memory *guaranteed* to be freed right after the 'del'
> > statement? Or should one call gc.collect to be really sure?
> > 
> > rainydayfund = [[] for x in xrange(16*1024)] # or however much you need
> > def handle_exception(e):
> > global rainydayfund
> > del rainydayfund
> > ... etc, etc ...
> > http://stackoverflow.com/questions/1235349/python-how-can-i-handle-any-unhandled-exception-in-an-alternative-way
> 
> I must admit that this looks rather strange to me, but I don't see any 
> problem why it wouldn't work. gc.collect() only helps with circular 
> dependencies, and there aren't any in your "rainy day fund".
> 
> Regarding the use of sys.excepthook I'd rather (SO voting be damned!) take 
> the obvious approach as suggested by Vinay Sajip, i. e.
> 
> try:
>     main()
> except:
>     # do what you have to do
> 
> instead of rewriting the hook. If potential memory resource hogs are 
> confined within main() with no references from any module namespace you 
> should have enough memory availaible to run the except suite without the 
> need for a rainy day fund. If you know about a specific global name that 
> references a consumer of a lot of memory, an in-memory db, say, then why not 
> handle the memory error there or at least use it in lieu of a dedicated 
> dummy? I. e.
> 
> in_memory_db = None
> try:
>     try:
>         main()
>     finally:
>         del in_memory_db
> except:
>     # do what you have to do

Hi all,

Thanks for your replies. Initially my main concern was to also log fatal/unhandled exceptions. The database MemoryError was also I was also an issue I was just facing, so Martelli's approach seemed nice. Meanwhile I found out I did not only delete "TOP 100" (which I used to debug my code) in my SELECT query, but also DISTINCT. After I corrected this MemoryErrors are no longer an issue with this function. :-) Still, I always wonder how close I am to a MemoryError (I am on a tiny Win 7 32 VM, with Python 2.7). I sometimes check the Task Manager to see how much RAM is in use. Is there a Python way to do something similar?

Best wishes,
Albert-Jan