[New-bugs-announce] [issue42873] Exponential time and space requirements for compilation of nested try/finally blocks

[Mark Dickinson]

New submission from Mark Dickinson <dickinsm at gmail.com>:

tl;dr - contrived (but relatively short) code involving nested try/finally blocks can produce disproportionately large bytecode. I'm not expecting or suggesting any action here, but the situation seemed at least worth noting. Feel free to close this issue as a "well don't do that, then" (a.k.a. "wont fix")

Longer: Python 3.9 changed the way that bytecode was generated for try/finally (see #33387). For a "try" block body that can do any of raise, return or fall-off-the-end-of-the-block, the corresponding finally block gets three separate paths in the bytecode. If such trys are nested <n> times, we get 3^n separate paths in the bytecode.

Example code:

----------------
def f():
    try:
        if something(): return
    finally:
        try:
            if something(): return
        finally:
            try:
                if something(): return
            finally:
                try:
                    if something(): return
                finally:
                    try:
                        if something(): return
                    finally:
                        do_cleanup()


import dis
dis.dis(f)
----------------

On my machine, running this and counting the do_cleanup invocations gives, as expected, a result of 243 = 3**5

% python3.9 nested_finally.py | grep do_cleanup | wc -l 
     243

That's fairly benign, but if I scale up to 10 nested blocks, the dis.dis call takes over 10 minutes to complete (the compilation itself still only takes a fraction of a second). The bytecode object is correspondingly large:

>>> len(f.__code__.co_code)
1741356

With 15 levels of nesting, compilation takes several seconds, and
the generated code is (again as expected) a couple of orders of magnitude larger:

>>> len(f.__code__.co_code)
533859040

I didn't try pushing this further than 15 levels of nesting.

As I said above, it's not clear to me whether this is actually an issue that needs to be addressed in practice. It seems unlikely that "real code" :TM: would run into this, but the effect seemed worth noting.

----------
components: Interpreter Core
messages: 384718
nosy: Mark.Shannon, mark.dickinson
priority: normal
severity: normal
status: open
title: Exponential time and space requirements for compilation of nested try/finally blocks
type: performance
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42873>
_______________________________________