setrecursionlimit

[Ned Batchelder]

On Wednesday, May 18, 2016 at 12:11:25 PM UTC-4, Steven D'Aprano wrote:
> The documentation for setrecursion limit warns against setting the limit too
> high:
> 
>     [quote]
>     The highest possible limit is platform-dependent. A user may need to
>     set the limit higher when they have a program that requires deep
>     recursion and a platform that supports a higher limit. This should
>     be done with care, because a too-high limit can lead to a crash.
>     [end quote]
> 
> https://docs.python.org/3/library/sys.html#sys.setrecursionlimit
> 
> Indeed, if you set the recursion limit too high, you can smash the memory
> heap and get a segfault. How exactly does that work?
> 
> Why doesn't setrecursionlimit() raise an exception when you try to set it
> too high? For example:
> 
> sys.setrecursionlimit(2000000000)
> 
> succeeds on my system, even though that's a ludicrously high number. (It is
> more than half the number of bytes of memory my computer has.)
> 
> 
> So why can't Python tell if I'm setting the limit too high?
> 
> (I'm assuming that if it could, it would.)

I believe the issue here is that Python recursion results in the C stack
growing.  Each Python function call creates a number of C function calls,
which grows the C stack.  The crash you see is the C stack overflowing.

Is there a way to know how large the C stack can grow, and how much it
will grow for each Python function call? That sounds complicated to get
right.

--Ned.