[Python-Dev] with_traceback

[Phillip J. Eby]

At 03:38 PM 2/26/2007 -0700, Andrew Dalke wrote:
>Guido's talk at PyCon said:
>
> >  Use raise E(arg).with_traceback(tb)
> >      instead of raise E, arg, tb
>
>That seems strange to me because of the mutability.  Looking through
>the back discussions on this list I see Guido commented:
>  http://mail.python.org/pipermail/python-3000/2007-February/005689.html
>
> > Returning the mutated object is acceptable here
> > because the *dominant* use case is creating and raising an exception
> > in one go:
> >
> >  raise FooException(<args>).with_traceback(<tb>)
>
>The 3 argument raise statement is rarely used, in my experience.
>I believe most don't even know it exists, excepting mostly advanced
>Python programmers and language lawyers.
>
>My concern when I saw Guido's keynote was the worry that
>people do/might write code like this
>
>NO_END_OF_RECORD = ParserError("Cannot find end of record")
>
>def parse_record(input_file):
>    ...
>     raise NO_END_OF_RECORD
>    ...
>
>
>That is, create instances at the top of the module, to be used
>later.  This code assume that the NO_END_OF_RECORD
>exception instance is never modified.
>
>If the traceback is added to its __traceback__ attribute then
>I see two problems if I were to write code like the above:
>
>   - the traceback stays around "forever"
>   - the code is no longer thread-safe.

Then don't do that, as it's bad style for Python 3.x.  ;-)

But do note that 3-argument raise should NOT be implemented this way in 
Python 2.x.  2.6 and other 2.x revisions should still retain the existing 
raise machinery, it's just that *catching* an exception using 3.x style 
("except foo as bar:") should call with_traceback() at the time of the catch.

This does mean you won't be able to port your code to 3.x style until 
you've gotten rid of shared exception instances from all your dependencies, 
but 3.x porting requires all your dependencies to be ported anyway.

It should be sufficient in both 2.x and 3.x for with_traceback() to raise 
an error if the exception already has a traceback -- this should catch any 
exception instance reuse.


>What is the correct way to rewrite this for use
>with "with_traceback"?  Is it
>
>def open_file_on_path(name):
>   # If nothing exists, raises an exception based on the
>   # first attempt
>   saved_err = None
>   for dirname in _PATH:
>     try:
>       return open(os.path.join(dirname, name))
>     except Exception, err:
>       if not saved_err:
>         saved_err = err
>         saved_tb = sys.exc_info()[2]
>   raise saved_err.with_traceback(saved_err.__traceback__)

No, it's more like this:

     try:
         for dirname in ...
             try:
                 return ...
             except Exception as err:
                saved_err = err
         raise saved_err
     finally:
         del saved_err

I've added the outer try-finally block to minimize the GC impact of the 
*original* code you showed, as the `saved_tb` would otherwise have created 
a cycle.  That is, the addition is not because of the porting, it's just 
something that you should've had to start with.

Anyway, the point here is that in 3.x style, most uses of 3-argument raise 
just disappear altogether.  If you hold on to an exception instance, you 
have to be careful about it for GC, but no more so than in current Python.

The "save one instance and use it forever" use case is new to me - I've 
never seen nor written code that uses it before now.  It's definitely 
incompatible with 3.x style, though.