[SciPy-Dev] Circular reference in interp1d causing memory errors

[josef.pktd at gmail.com]

On Sat, May 18, 2013 at 8:17 AM, Robert Kern <robert.kern at gmail.com> wrote:

> On Sat, May 18, 2013 at 12:14 PM,  <josef.pktd at gmail.com> wrote:
>
> > interp._call is an instance method assigned in the __init__
> > This is a pattern we use quite a few times (stats.distribution does it
> > a lot, statsmodels in some models).
> >
> > I know we have problems with pickling in these cases, but didn't know
> > about the garbage collection.
> >
> > I didn't find much explicit information after googling for an hour.
> >
> http://stackoverflow.com/questions/9329232/internal-reference-prevents-garbage-collection
> > looks similar but doesn't use create instance methods.
>
> Yes, this is the same issue, just storing the bound methods in a dict
> rather than directly as an attribute on the instance.
>
> > The solution
> > with calling through the class might be interesting in some cases.
> > I found a few more threads on various mailing lists but never a clear
> > answer or link.
> >
> > Does anyone know a more explicit explanation?
> > Or, is it really just the cyclical reference, even though it's a
> > method (which always hold self)
>
> The bound method is not created with the .im_self reference until it
> is requested. Until then, the *unbound* method lives on the class
> without a reference to any particular instance.
>
> [~]
> |1> from scipy.interpolate import interp1d
>
> [~]
> |2> f = interp1d(np.linspace(0,1), np.linspace(0,1))
>
> [~]
> |3> f._call_linear is f._call_linear
> False
>
>
> Requesting the method creates a new bound method that refers to the
> instance.
>
> [~]
> |4> f._call_linear.im_self
> <scipy.interpolate.interpolate.interp1d at 0x10b4f8710>
>
> [~]
> |5> f
> <scipy.interpolate.interpolate.interp1d at 0x10b4f8710>
>
>
> Storing that newly created bound method as a new attribute on itself
> creates a cycle.
>

Thank, took me a while but I think I understand now

>>> f._call is f._call
True
>>> f._call_linear is f._call_linear
False

I didn't understand or looked at this part of the python documentation
before (which I guess is the relevant part):
"Note that the transformation from function object to (unbound or bound)
method object happens each time the attribute is retrieved from the class
or instance. In some cases, a fruitful optimization is to assign the
attribute to a local variable and call that local variable. "

Josef


>
> --
> Robert Kern
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20130518/f643fa69/attachment.html>