[Python-ideas] Shrink recursion error tracebacks (was: Have REPL print less by default)

[Steven D'Aprano]

On Sat, Apr 23, 2016 at 02:14:10AM -0400, Franklin? Lee wrote:

> Here's pseudocode for my suggestion.
> (I assume appropriate definitions of `traceback`, `print_folded`, and
> `print_the_thing`. I assume `func_name` is a qualified name (e.g.
> `('f', '<stdin>')`).)
> 
>     seen = collections.Counter() # Counts number of times each
> (func,line_no) has been seen
>     block = None # A block of potentially-hidden functions.
>     prev_line_no = None # In case len(block) == 1.
>     hidecount = 0 # Total number of hidden lines.
>     for func_name, line_no in traceback:
>         times = seen[func_name, line_no] += 1
>         if times >= 3:
>             if block is None:
>                 block = collections.Counter()
>             block[func_name] += 1
>             prev_line_no = line_no
>         else:
>             # This `if` can be a function which returns a hidecount,
>             #  so we don't repeat ourselves at the end of the loop.
>             if block is not None:
>                 if len(block) == 1: # don't need to hide
>                     print_the_thing(next(block.
> keys()), prev_line_no)
>                 else:
>                     print_folded(block)
>                     hidecount += len(block)
>                 block = None
>             print_the_thing(func_name, line_no)
> 
>     if block is not None:
>         if len(block) == 1:
>             print_the_thing(block[0])
>         else:
>             print_folded(block)
>             hidecount += len(block)



Just in case anyone missed it, here's my actual, working, code, which I 
now have in my PYTHONSTARTUP file.


import sys
import traceback
from itertools import groupby
TEMPLATE = "  [...previous call is repeated %d times...]\n"

def collapse(seq):
    for key, group in groupby(seq):
        group = list(group)
        if len(group) < 3:
            for item in group:
                yield item
        else:
            yield key
            yield TEMPLATE % (len(group)-1)

def shortertb(*args):
    lines = traceback.format_exception(*args)
    sys.stderr.write(''.join(collapse(lines)))

sys.excepthook = shortertb


And here is an actual working example of it in action:

py> import fact  # Uses the obvious recursive algorithm.
py> fact.fact(50000)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/steve/python/fact.py", line 3, in fact
    return n*fact(n-1)
  [...previous call is repeated 997 times...]
  File "/home/steve/python/fact.py", line 2, in fact
    if n < 1: return 1
RuntimeError: maximum recursion depth exceeded in comparison


I'm not very interested in a complex, untested, incomplete, non-working 
chunk of pseudo-code when I have something which actually works in less 
than twenty lines. Especially since your version hides useful traceback 
information and requires the user to call a separate function to display 
the unmangled (and likely huge) traceback to find out what they're 
missing.

In my version, they're not missing anything: it's a simple run-length 
encoding of the tracebacks, and nothing is hidden. The output is just 
compressed.

So I'm afraid that, even if you manage to get your pseudo-code working 
and debugged, I'm going to vote a strong -1 on your proposal. Even if it 
works, I don't want lines to be hidden just because they've been seen 
before in some unrelated part of the traceback.

> My hiding is more complex (can't reproduce original output exactly),
> so it would be important to have an obvious way to get the old
> behavior. Someone else can propose the wording, if the hiding strategy
> itself seems useful.

To me, it seems harmful, not useful.


-- 
Steve