[New-bugs-announce] [issue37213] Peeephole optimizer does not optimize functions with multiline expressions

[Serhiy Storchaka]

New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

The optimization is skipped if lnotab contains 255. It was very uncommon in older versions (only when the function contains very large expressions, larger than hundreds of lines or bytecode instructions), but in 3.8 this situation is common.

For example:

[x
 for x in a if x]

  1           0 BUILD_LIST               0
              2 LOAD_FAST                0 (.0)
        >>    4 FOR_ITER                12 (to 18)

  2           6 STORE_FAST               1 (x)
              8 LOAD_FAST                1 (x)
             10 POP_JUMP_IF_FALSE       16

  1          12 LOAD_FAST                1 (x)
             14 LIST_APPEND              2
        >>   16 JUMP_ABSOLUTE            4
        >>   18 RETURN_VALUE

if x:
    if (y and
        z):
        foo()
else:
    bar()

  1           0 LOAD_NAME                0 (x)
              2 POP_JUMP_IF_FALSE       20

  2           4 LOAD_NAME                1 (y)
              6 POP_JUMP_IF_FALSE       18

  3           8 LOAD_NAME                2 (z)

  2          10 POP_JUMP_IF_FALSE       18

  4          12 LOAD_NAME                3 (foo)
             14 CALL_FUNCTION            0
             16 POP_TOP
        >>   18 JUMP_FORWARD             6 (to 26)

  6     >>   20 LOAD_NAME                4 (bar)
             22 CALL_FUNCTION            0
             24 POP_TOP
        >>   26 LOAD_CONST               0 (None)
             28 RETURN_VALUE

You can see non-optimized jumps to jumps (from 10 to 16 and from 6 and 10 to 16 correspondingly).

This is a consequence of two features: ability to encode negative line differences in lnotab and setting lines for both outer and inner expressions.

Two ways to solve this issue:

1. Move optimizations from Python/peephole.c to Python/compile.c (see issue32477 and issue33318). This is a new feature and it is too late for 3.8.

2. Make the peepholer to work with lnotab containing 255.

Pablo, are you interesting?

----------
components: Interpreter Core
messages: 345108
nosy: benjamin.peterson, pablogsal, serhiy.storchaka, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: Peeephole optimizer does not optimize functions with multiline expressions
type: behavior
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37213>
_______________________________________