[issue34214] Pylint recusion stack overflow abort

[Nick Drozd]

New submission from Nick Drozd <nicholasdrozd at gmail.com>:

In certain very, very specific circumstances, Pylint can cause the interpreter to abort with a stack overflow error.

I've seen this issue on 3.5, 3.6, and 3.8 (I assume it affects 3.7 as well) and on both Mac and Ubuntu.

It requires:
  * 1.7 <= Pylint version < 1.9
  * 1.6.0 <= Astroid version < 1.6.2
    - (For interpreter versions > 3.6, commit https://github.com/PyCQA/astroid/commit/ceeee09743079d07ffc3918969ab203f766b29e0 is also required.)
  * The dependencies of Pylint and Astroid (I'm not aware of any specific versions needed.)

The crash comes from running Pylint on the file stack-overflow-abort.py (https://github.com/nickdrozd/pylint/blob/80b05dace5049b8a837b8c530a57321381717ddb/stack-overflow-abort.py). This causes the interpreter to abort with the message "Fatal Python error: Cannot recover from stack overflow."

Now, it's not surprising that some kind of stack depth error occurs here, as the target code is recursive, Pylint is recursive, and and Astroid 1.6.0 had a bug in its handling of augmented assign statements (this bug was fixed in 1.6.2 with a one-line change: https://github.com/PyCQA/astroid/commit/3fae32f99c7f432a1281b151ea17e163a2e8dc47). But this recursion bug should be handled with a RecursionError instead of an abort.

What's strange is that running Pylint on the file caught-recursion-error.py (https://github.com/nickdrozd/pylint/blob/80b05dace5049b8a837b8c530a57321381717ddb/caught-recursion-error.py), which differs from stack-overflow-abort.py in just three characters, does end up with a RecursionError (neither file is large at all). So it looks like something subtle is happening here. I tried looking around to see if I could figure it out, and I could not. All I was able to discover is that changing 50 to other numbers in _Py_CheckRecursiveCall causes the error to come from different places: at the beginning of the eval loop, "while calling a Python object" at several points, and "in comparison".

I've included the text of a shell session showing the errors.

Repo links to versions of Pylint and Astroid that cause the error:
https://github.com/nickdrozd/pylint/tree/crash
https://github.com/nickdrozd/astroid/tree/crash

See https://github.com/PyCQA/pylint/issues/1805 for further discussion.

----------
components: Interpreter Core
files: shell-session
messages: 322328
nosy: nickdrozd
priority: normal
severity: normal
status: open
title: Pylint recusion stack overflow abort
type: crash
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47711/shell-session

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34214>
_______________________________________