[ python-Bugs-1469629 ] __dict__ = self in subclass of dict causes a memory leak?

[SourceForge.net]

Bugs item #1469629, was opened at 2006-04-12 22:04
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1469629&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Dobes V (dobesv)
Assigned to: Nobody/Anonymous (nobody)
Summary: __dict__ = self in subclass of dict causes a memory leak?

Initial Comment:

Using:

ActivePython 2.4.2 Build 10 (ActiveState Corp.) based 
on
Python 2.4.2 (#67, Jan 17 2006, 15:36:03) [MSC v.1310 
32 bit (Intel)] on win32

For reasons I do not understand, the following class 
leaks itself continuously:

class AttrDict(dict):
   def __init__(self, *args, **kw):
      dict.__init__(self, *args, **kw)
      self.__dict__ = self

Whereas this version does not:

class AttrDict(dict):
   def __init__(self, *args, **kw):
      dict.__init__(self, *args, **kw)
   
   def __getattr__(self, key):
      return self[key]
   
   def __setattr__(self, key, value):
      self[key] = value

My test looks like this:

for n in xrange(1000000):
   import gc
   gc.collect()
   ad = AttrDict()
   ad['x'] = n
   ad.y = ad.x
   print n, ad.x, ad.y

And I sit and watch in the windows task manager while 
the process grows and grows.  With the __getattr__ 
version, it doesn't grow.




----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2006-04-14 00:19

Message:
Logged In: YES 
user_id=33168

Armin, why not just check any leaking test cases into
Lib/test/leakers?  I have no idea if your patch is correct
or not.  I wouldn't be surprised if there are a bunch more
issues similar to this.  What if you stick self in
self.__dict__ (I'm guessing this is ok, but there are a
bunch of variations) or start playing with weakrefs?

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2006-04-13 02:58

Message:
Logged In: YES 
user_id=4771

This is caused by the tp_clear not doing its job -- 
in this case, tp_clear is subtype_clear(), which does
not reset the __dict__ slot to NULL because it assumes
that the __dict__ slot's content itself will be cleared,
which is perfectly true but doesn't help if self.__dict__
is self.

Attached a patch to fix this.  It's kind of hard to
test for this bug because all instances of AttrDict
are really cleared, weakrefs to them are removed,
etc.

Also attached is an example showing a similar bug: a
cycle through the ob_type field, with a object U
whose class is itself. It is harder to clear this
link because we cannot just set ob_type to NULL in
subtype_clear.  Ideas welcome...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1469629&group_id=5470