[Python-Dev] with_traceback

[Andrew Dalke]

Guido's talk at PyCon said:

>  Use raise E(arg).with_traceback(tb)
>      instead of raise E, arg, tb

That seems strange to me because of the mutability.  Looking through
the back discussions on this list I see Guido commented:
 http://mail.python.org/pipermail/python-3000/2007-February/005689.html

> Returning the mutated object is acceptable here
> because the *dominant* use case is creating and raising an exception
> in one go:
>
>  raise FooException(<args>).with_traceback(<tb>)

The 3 argument raise statement is rarely used, in my experience.
I believe most don't even know it exists, excepting mostly advanced
Python programmers and language lawyers.

My concern when I saw Guido's keynote was the worry that
people do/might write code like this

NO_END_OF_RECORD = ParserError("Cannot find end of record")

def parse_record(input_file):
   ...
    raise NO_END_OF_RECORD
   ...


That is, create instances at the top of the module, to be used
later.  This code assume that the NO_END_OF_RECORD
exception instance is never modified.

If the traceback is added to its __traceback__ attribute then
I see two problems if I were to write code like the above:

  - the traceback stays around "forever"
  - the code is no longer thread-safe.


As an example of code which is affected by this, see pyparsing,
which has code that looks like

class Token(ParserElement):
    """Abstract ParserElement subclass, for defining atomic matching patterns.""
"
    def __init__( self ):
        super(Token,self).__init__( savelist=False )
        self.myException = ParseException("",0,"",self)

     ....
class Literal(Token):
    ....
    def parseImpl( self, instring, loc, doActions=True ):
        if (instring[loc] == self.firstMatchChar and
            (self.matchLen==1 or instring.startswith(self.match,loc)) ):
            return loc+self.matchLen, self.match
        #~ raise ParseException( instring, loc, self.errmsg )
        exc = self.myException
        exc.loc = loc
        exc.pstr = instring
        raise exc

The "Literal" and other token classes are part of the
grammar definition and usually exist in module scope.



There's another question I came up with.  If the exception
already has a __traceback__, will that traceback be
overwritten if the instance is reraised?  Consider this code,
loosly derived from os._execvpe

import os, sys
_PATH = ["here", "there", "elsewhere"]

def open_file_on_path(name):
  # If nothing exists, raises an exception based on the
  # first attempt
  saved_err = None
  saved_tb = None
  for dirname in _PATH:
    try:
      return open(os.path.join(dirname, name))
    except Exception, err:
      if not saved_err:
        saved_err = err
        saved_tb = sys.exc_info()[2]
  raise saved_err.__class__, saved_err, saved_tb

open_file_on_path("spam")

which generates this

Traceback (most recent call last):
  File "raise.py", line 19, in <module>
    open_file_on_path("spam")
  File "raise.py", line 11, in open_file_on_path
    return open(os.path.join(dirname, name))
IOError: [Errno 2] No such file or directory: 'here/spam'

What is the correct way to rewrite this for use
with "with_traceback"?  Is it

def open_file_on_path(name):
  # If nothing exists, raises an exception based on the
  # first attempt
  saved_err = None
  for dirname in _PATH:
    try:
      return open(os.path.join(dirname, name))
    except Exception, err:
      if not saved_err:
        saved_err = err
        saved_tb = sys.exc_info()[2]
  raise saved_err.with_traceback(saved_err.__traceback__)


One alternative, btw, is
  raise saved_err.with_traceback()

to have it use the existing __traceback__ (and raising its
own exception if __traceback__ is None?)

        Andrew
        dalke at dalkescientific.com