[IPython-dev] Getting InteractiveShell to clean up after itself

[Fernando Perez]

Howdy,

On Wed, Aug 26, 2009 at 11:19 AM, Brian Granger<ellisonbg.net at gmail.com> wrote:
> Hi,
>
> Currently in my ipython-app branch, InteractiveShell is a real object that
> can be instantiated:
>
> ip = InteractiveShell()
>
> But, when you do:
>
> del ip
> gc.collect()
>
> The InteractiveShell.__del__ does *not* get called.  This is not too
> surprising given how garbage collection
> works in python:
>
> *  We have cycles to InteractiveShell all over the place (everybody points
> to it!)
> *  In Python cycles + __del__ methods => some objects can't be collected
> even when gc.collect() is run.
>
> I would like to resolve this problem and I see two ways of accomplishing it:
>
> 1.  Begin to use weakrefs everywhere.  This would break all our cycles and
> would probably be a good idea
> anyways.  If we do this with discipline, we might be able to get a __del__
> method called to do cleanup
> (removing sys.*hook, removing things injected into __builtin__, etc.).
>
> 2.  Add a .cleanup() method that does all the cleanup by hand.  This won't
> break cycles, but it would ensure
> that the InteractiveShell had "detached" from everything outside of itself.
> If we go this way, and drop 2.4
> support, we could make InteractiveShell a context manager and have cleanup
> called when the context
> exists.
>
> My goal is that we could create and destroy InteractiveShell instances (for
> testing, etc.) whenever we want
> or need and they would always clean up after themselves.
>
> If we don't go this route, then I think we should make InteractiveShell a
> singleton.
>
> Thoughts?

I need to run now, so rather hastily:

1. While I'm all for using more weakrefs, they are a subtle tool and I
doubt we can get away with using them everywhere for everything.  I
don't want to 'bet the farm' on that strategy, because we can easily
find a situation where weakrefs don't work, we have way too much
user-facing  state managemnet we must accomplish robustly and weakrefs
may not be the tool for everything we need to do.

2. This is my favorite.  I wouldn't make the shell necessarily a
context manager by itself yet, I'm not sure it's the right interface
(I tend to prefer objects that have small, well-defined interfaces by
themselves), but it would be easy, and good, to have a context manager
that manages a shell instance for us and cleans it up on exit.

How does this sound?

Cheers,
f