[Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup

[Phillip J. Eby]

At 10:09 AM 5/19/2005 -0700, Guido van Rossum wrote:
>[Phillip J. Eby]the only way a generator-iterator can become
> > part of a cycle is via  an action taken outside the generator.  (E.g.
> > passing it into itself via 'continue', creating a link from one of its
> > arguments to it, etc.)  So, it's probably not a terrible limitation in
> > practice.
>
>It's enough to store a reference to the generator in a global (or in
>anything that's reachable from a global). The generator's frame's
>f_globals pointer then ensures the cycle.

Well, if the generator was defined in the same module, I suppose.  This 
would probably only happen with short scripts, where the lack of GC is 
unlikely to be an issue.

However, at least it would also be possible to explicitly close the 
generator, which wasn't possible before.


>Throwing an exception also provides ample opportunity for creating
>cycles, since the frame hold a reference to the most recent traceback.
>Ironically, throwing an exception with a traceback into a generator is
>likely to cause a cycle because the traceback likely references the
>throwing frame, which certainly has a reference to the generator...

*head exploding*  Double ouch.

Wait a minute...  those cycles don't include the generator, do they?  Let 
me think.  Frame A has a reference to the generator iterator, and invokes 
throw() (directly or indirectly) on it.  Frame B, the generator frame, gets 
its f_back set to point to Frame A, but presumably that link is cleared on 
exit?  (If it isn't, it probably should be).

Anyway, frame B throws an exception, and the traceback is created.  The 
traceback has a reference to frame B.  We return to frame A, and add it to 
the traceback as well, and a reference to the traceback goes into the frame 
too.  Hm.  Still no cycle passing through the *generator iterator*, unless 
the generator's frame's f_back is still pointing to the frame that it was 
last called from.  This holds even if the generator's frame holds the 
traceback that was current at the time of the error, because that traceback 
only includes the generator's frame, not the caller's frame.

So, as long as the generator guarantees its frame's f_back is empty while 
the generator is not actually executing, the cycle should not include the 
generator, so it will still be GC'able.  (Note, by the way, that this 
property is probably another good reason for using the yield expression as 
the source location for a throw()!)