[New-bugs-announce] [issue38593] Python 3.7 does not catch infinite recursion for some values of sys.getrecursionlimit()

[Valentyn Tymofieiev]

New submission from Valentyn Tymofieiev <valentyn at google.com>:

While investigating an issue[1] in Apache Beam, we observed that Python 3.7 does not catch an infinite recursion for some values of sys.getrecursionlimit().

Repro steps:

docker run -it --entrypoint=/bin/bash python:3.7-stretch
git clone https://github.com/tvalentyn/beam.git
cd beam
git checkout origin/reproduce_uncaught_recursion_error
cd sdks/python
pip install nose
pip install -e .[gcp,test]
python ./setup.py nosetests --nocapture --tests 'apache_beam/runners/dataflow/dataflow_runner_test.py:DataflowRunnerTest.test_remote_runner_display_data'

On Python 3.7, the test output is:

==========
Recursion limit set to: 1002
Caught maximum recursion depth exceeded while calling a Python object. Current recursionlimit is 1002
Recursion limit set to: 1003
Caught maximum recursion depth exceeded. Current recursionlimit is 1003
Recursion limit set to: 1004
Caught maximum recursion depth exceeded while calling a Python object. Current recursionlimit is 1004
Recursion limit set to: 1005
Caught maximum recursion depth exceeded while calling a Python object. Current recursionlimit is 1005
Recursion limit set to: 1006
Fatal Python error: Cannot recover from stack overflow.

Current thread 0x00007fd9f28a5700 (most recent call first):
  File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 424 in get
...
==========

So, if we set recursion limit to 1006 in [2], Python 3.7 interpreter does not throw/catch RecursionError in [3]. On Python 3.5, 3.6 the same code successfully catches RecursionError.  
 
[1] https://issues.apache.org/jira/browse/BEAM-8397 
[2] https://github.com/tvalentyn/beam/blob/158cc9006fef74fc17d52a4220172758f7a82820/sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py#L235 
[3] https://github.com/tvalentyn/beam/blob/158cc9006fef74fc17d52a4220172758f7a82820/sdks/python/apache_beam/pipeline.py#L599

----------
components: Interpreter Core
messages: 355389
nosy: Valentyn Tymofieiev
priority: normal
severity: normal
status: open
title: Python 3.7 does not catch infinite recursion for some values of sys.getrecursionlimit()
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38593>
_______________________________________