How to pop the interpreter's stack?

[Steven D'Aprano]

On Thu, 16 Dec 2010 10:39:34 -0600, Robert Kern wrote:

> On 12/16/10 10:23 AM, Steven D'Aprano wrote:
>> On Thu, 16 Dec 2010 07:29:25 -0800, Ethan Furman wrote:
>>
>>> Tim Arnold wrote:
>>>> "Ethan Furman"<ethan at stoneleaf.us>  wrote in message
>>>> news:mailman.4.1292379995.6505.python-list at python.org...
>>>>> kj wrote:
>>>>>> The one thing I don't like about this strategy is that the
>>>>>> tracebacks of exceptions raised during the execution of __pre_spam
>>>>>> include one unwanted stack level (namely, the one corresponding to
>>>>>> __pre_spam itself).
>> [...]
>>> A decorator was one of the items kj explicity didn't want.  Also,
>>> while it would have a shallower traceback for exceptions raised during
>>> the __pre_spam portion, any exceptions raised during spam itself would
>>> then be one level deeper than desired... while that could be masked by
>>> catching and (re-?)raising the exception in the decorator, Steven had
>>> a very good point about why that is a bad idea -- namely, tracebacks
>>> shouldn't lie about where the error is.
>>
>> True, very true... but many hours later, it suddenly hit me that what
>> KJ was asking for wasn't *necessarily* such a bad idea. My thought is,
>> suppose you have a function spam(x) which raises an exception. If it's
>> a *bug*, then absolutely you need to see exactly where the error
>> occurred, without the traceback being mangled or changed in any way.
>>
>> But what if the exception is deliberate, part of the function's
>> documented behaviour? Then you might want the exception to appear to
>> come from the function spam even if it was actually generated inside
>> some private sub-routine.
> 
> Obfuscating the location that an exception gets raised prevents a lot of
> debugging (by inspection or by pdb), even if the exception is
> deliberately raised with an informative error message. Not least, the
> code that decides to raise that exception may be buggy. But even if the
> actual error is outside of the function (e.g. the caller is passing bad
> arguments), you want to at least see what tests the __pre_spam function
> is doing in order to decide to raise that exception.

And how do you think you see that from the traceback? The traceback 
prints the line which actually raises the exception (and sometimes not 
even that!), which is likely to be a raise statement:

>>> import example
>>> example.func(42)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "example.py", line 3, in func
    raise ValueError('bad value for x')
ValueError: bad value for x

The actual test is:

def func(x):
    if x > 10 and x%2 == 0:
        raise ValueError('bad value for x')

but you can't get that information from the traceback.

Python's exception system has to handle two different situations: buggy 
code, and bad data. It's not even clear whether there is a general 
distinction to be made between the two, but even if there's not a general 
distinction, there's certainly a distinction which we can *sometimes* 
make. If a function contains a bug, we need all the information we can 
get, including the exact line that causes the fault. But if the function 
deliberately raises an exception due to bad input, we don't need any 
information regarding the internals of the function (assuming that the 
exception is sufficiently detailed, a big assumption I grant you!). If I 
re-wrote the above func() like this:

def func(x):
    if !(x <= 10):
        if x%2 != 0: 
            pass
        else:
            raise ValueError('bad value for x')
    return

I would have got the same traceback, except the location of the exception 
would have been different (line 6, in a nested if-block). To the caller, 
whether I had written the first version of func() or the second is 
irrelevant. If I had passed the input validation off to a second 
function, that too would be irrelevant.

I don't expect Python to magically know whether an exception is a bug or 
not, but there's something to be said for the ability to turn Python 
functions into black boxes with their internals invisible, like C 
functions already are. If (say) math.atan2(y, x) raises an exception, you 
have no way of knowing whether atan2 is a single monolithic function, or 
whether it is split into multiple pieces. The location of the exception 
is invisible to the caller: all you can see is that atan2 raised an 
exception.


> Tracebacks are inherently over-verbose. This is necessarily true because
> no algorithm (or clever programmer) can know all the pieces of
> information that the person debugging may want to know a priori. Most
> customizations of tracebacks *add* more verbosity rather than reduce it.
> Removing one stack level from the traceback barely makes the traceback
> more readable and removes some of the most relevant information.

Right. But I have thought of a clever trick to get the result KJ was 
asking for, with the minimum of boilerplate code. Instead of this:


def _pre_spam(args):
    if condition(args):
        raise SomeException("message")
    if another_condition(args):
        raise AnotherException("message")
    if third_condition(args):
        raise ThirdException("message")

def spam(args):
    _pre_spam(args)
    do_useful_work()


you can return the exceptions instead of raising them (exceptions are 
just objects, like everything else!), and then add one small piece of 
boilerplate to the spam() function:


def _pre_spam(args):
    if condition(args):
        return SomeException("message")
    if another_condition(args):
        return AnotherException("message")
    if third_condition(args):
        return ThirdException("message")

def spam(args):
    exc = _pre_spam(args)
    if exc: raise exc
    do_useful_work()




-- 
Steven