[issue37166] inspect.findsource doesn't handle shortened files gracefully

[Tim Hatch]

New submission from Tim Hatch <tim at timhatch.com>:

inspect.findsource() can trigger IndexError when co_firstlineno is larger than len(linecache.getlines()).

If you have a situation where the file that linecache finds doesn't match the imported module, then you're not guaranteed that co_firstlineno on the code objects makes any sense.  We hit this in a special kind of par file, but it could be triggered by shortening the file, like doing a git checkout of an older version while it's running.

a.py (3 lines):
    # line 1
    # line 2
    def foo(): pass  # co_firstlineno=3

a.py.v2 (1 line):
    def foo(): pass

repro:
    import a
    # modification happens, cp a.py.v2 a.py
    inspect.getsource(a.foo)


Although linecache.getline() does bounds checking, `inspect.findsource()` (which is used by various other functions, including `inspect.stack()`) grabs all the lines and loops through them.  The bug is in this section of `inspect.py`:

    if iscode(object):
        if not hasattr(object, 'co_firstlineno'):
            raise OSError('could not find function definition')
        lnum = object.co_firstlineno - 1
        pat = re.compile(r'^(\s*def\s)|(\s*async\s+def\s)|(.*(?<!\w)lambda(:|\s))|^(\s*@)')
        while lnum > 0:
            if pat.match(lines[lnum]): break
            lnum = lnum - 1
        return lines, lnum
    raise OSError('could not find code object')

Found through future.utils.raise_from which actually doesn't need to be using `inspect.stack()`, it can switch to `sys._getframe(2)` or so.  We should have a PR ready shortly, but wondering if this can be backported to at least 3.6?

----------
components: Library (Lib)
messages: 344761
nosy: lisroach, thatch, vstinner
priority: normal
severity: normal
status: open
title: inspect.findsource doesn't handle shortened files gracefully
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37166>
_______________________________________