[issue43674] strange effect at recursion limit

[Eryk Sun]

Eryk Sun <eryksun at gmail.com> added the comment:

> Why is there no 2nd 995 after the recursion error?

print(s) calls sys.stdout.write(s) and sys.stdout.write("\n"). The I/O stack consists of a text I/O wrapper, buffered writer, and a raw file object. What happens in this case is that writing "\n" causes the text wrapper to flush the buffer, which flushes to the raw file. Calling the latter's write() method raises a RecursionError. But "995\n" is still buffered and gets written out later on when the exception handler prints "recursion error".

Try replacing print() with low-level os.write(1, bytes_text). For example:

    import os

    def recurse(n):
        os.write(1, f'{n}\n'.encode())
        try:
            recurse(n + 1)
        except RecursionError:
            os.write(1, b'recursion error\n')
        os.write(1, f'{n}\n'.encode())

> Is there a way to determine the practically working maximum it can do?

The size of a C stack frame varies depending on the C function call, compiler optimizations, and the platform.

If you need to increase the recursion limit (not recommended) by a given factor, I suggest calling the recursive function in a new thread that has a stack size that's increased by the same factor over the default limit (depends on the platform). The stack size for new threads is set via old_size = threading.stack_size(new_size). Make sure that the already running threads (e.g. the main thread) that use the default stack size are executing code that stays well below the original recursion limit, else this could crash the interpreter.

In Windows, the default stack size is 2 MiB, and the maximum size that Python allows is one byte less than 256 MiB (268,435,455), which is about 128 times the default size. The largest size should safely (conservatively) support a recursion limit of 128,000 frames. With the above recurse() function, I tested that the maximum stack size crashes with a stack overflow at about 325,000 frames.

----------
nosy: +eryksun
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
versions: +Python 3.10 -Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43674>
_______________________________________