[Python-checkins] CVS: python/dist/src/Lib doctest.py,1.12,1.13

[Tim Peters]

Update of /cvsroot/python/python/dist/src/Lib
In directory usw-pr-cvs1:/tmp/cvs-serv4624/Lib

Modified Files:
	doctest.py 
Log Message:
Clear the copy of the globs dict after running examples.  This helps to
break cycles, which are a special problem when running generator tests
that provoke exceptions by invoking the .next() method of a named
generator-iterator:  then the iterator is named in globs, and the
iterator's frame gets a tracekback object pointing back to globs, and
gc doesn't chase these types so the cycle leaks.

Also changed _run_examples() to make a copy of globs itself, so its
callers (direct and indirect) don't have to (and changed the callers
to stop making their own copies); *that* much is a change I've been
meaning to make for a long time (it's more robust the new way).

Here's a way to provoke the symptom without doctest; it leaks at a
prodigious rate; if the last two "source" lines are replaced with
    g().next()
the iterator isn't named and then there's no leak:

source = """\
def g():
    yield 1/0

k = g()
k.next()
"""

code = compile(source, "<source>", "exec")

def f(globs):
    try:
        exec code in globs
    except ZeroDivisionError:
        pass

while 1:
    f(globals().copy())

After this change, running test_generators in an infinite loop still leaks,
but reduced from a flood to a trickle.


Index: doctest.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/doctest.py,v
retrieving revision 1.12
retrieving revision 1.13
diff -C2 -r1.12 -r1.13
*** doctest.py	2001/06/24 18:59:01	1.12
--- doctest.py	2001/06/24 20:02:47	1.13
***************
*** 530,538 ****
      return failures, len(examples)
  
! # Run list of examples, in context globs.  Return (#failures, #tries).
  
  def _run_examples(examples, globs, verbose, name):
      import sys
      saveout = sys.stdout
      try:
          sys.stdout = fakeout = _SpoofOut()
--- 530,542 ----
      return failures, len(examples)
  
! # Run list of examples, in a shallow copy of context (dict) globs.
! # Return (#failures, #tries).
! # CAUTION:  globs is cleared before returning.  This is to help break
! # cycles that may have been created by the examples.
  
  def _run_examples(examples, globs, verbose, name):
      import sys
      saveout = sys.stdout
+     globs = globs.copy()
      try:
          sys.stdout = fakeout = _SpoofOut()
***************
*** 541,544 ****
--- 545,555 ----
      finally:
          sys.stdout = saveout
+         # While Python gc can clean up most cycles on its own, it doesn't
+         # chase frame objects.  This is especially irksome when running
+         # generator tests that raise exceptions, because a named generator-
+         # iterator gets an entry in globs, and the generator-iterator
+         # object's frame's traceback info points back to globs.  This is
+         # easy to break just by clearing the namespace.
+         globs.clear()
      return x
  
***************
*** 546,550 ****
      """f, globs, verbose=0, name="NoName" -> run examples from f.__doc__.
  
!     Use dict globs as the globals for execution.
      Return (#failures, #tries).
  
--- 557,561 ----
      """f, globs, verbose=0, name="NoName" -> run examples from f.__doc__.
  
!     Use (a shallow copy of) dict globs as the globals for execution.
      Return (#failures, #tries).
  
***************
*** 736,740 ****
          e = _extract_examples(s)
          if e:
!             f, t = _run_examples(e, self.globs.copy(), self.verbose, name)
          if self.verbose:
              print f, "of", t, "examples failed in string", name
--- 747,751 ----
          e = _extract_examples(s)
          if e:
!             f, t = _run_examples(e, self.globs, self.verbose, name)
          if self.verbose:
              print f, "of", t, "examples failed in string", name
***************
*** 774,779 ****
          if self.verbose:
              print "Running", name + ".__doc__"
!         f, t = run_docstring_examples(object, self.globs.copy(),
!                                       self.verbose, name)
          if self.verbose:
              print f, "of", t, "examples failed in", name + ".__doc__"
--- 785,789 ----
          if self.verbose:
              print "Running", name + ".__doc__"
!         f, t = run_docstring_examples(object, self.globs, self.verbose, name)
          if self.verbose:
              print f, "of", t, "examples failed in", name + ".__doc__"