memory leak with dynamically defined functions?

[Skip Montanaro]

    zooko> In searching for the source of the leakge today, I encountered a
    zooko> surprising behaviour of Python which I think may be a bug.  If I
    zooko> allocate a bunch of memory with a function like this:

    >>> blarg = {}
    >>> for i in range(2**13):
    ...     blarg[i] = [0] * (2**10)

    zooko> and then remove the references to this memory, like this:

    >>> del blarg

    zooko> then all the memory is freed up.

    zooko> But if I allocate memory and store a reference to it in a default
    zooko> argument to an inner function, like this:

    >>> def silliest_func():
    >>>     x = [0] * (2**10)
    >>>     def inner_silliest_func(x=x):
    >>>         pass
    >>>     return inner_silliest_func
    >>> 
    >>> blarg = {}
    >>> for i in range(2**13):
    >>>     blarg[i] = silliest_func()

    zooko> and then remove the references to this memory, like this:

    >>> del blarg

    zooko> none of the memory is freed up!

    zooko> Even stimulating the garbage collector, with:

    >>> import gc
    >>> gc.collect()
    0

    zooko> just returns "0" and the memory is still in use.

The garbage collector only considers containers (lists, tuples, dicts, and a
couple other types perhaps) as potential sources of circularity.  I don't
think functions are considered.

-- 
Skip Montanaro (skip at pobox.com)
http://www.mojam.com/
http://www.musi-cal.com/