setrecursionlimit

[Rob Gaddi]

Ned Batchelder wrote:

> On Wednesday, May 18, 2016 at 12:11:25 PM UTC-4, Steven D'Aprano wrote:
>> The documentation for setrecursion limit warns against setting the limit too
>> high:
>> 
>>     [quote]
>>     The highest possible limit is platform-dependent. A user may need to
>>     set the limit higher when they have a program that requires deep
>>     recursion and a platform that supports a higher limit. This should
>>     be done with care, because a too-high limit can lead to a crash.
>>     [end quote]
>> 
>> https://docs.python.org/3/library/sys.html#sys.setrecursionlimit
>> 
>> Indeed, if you set the recursion limit too high, you can smash the memory
>> heap and get a segfault. How exactly does that work?
>> 
>> Why doesn't setrecursionlimit() raise an exception when you try to set it
>> too high? For example:
>> 
>> sys.setrecursionlimit(2000000000)
>> 
>> succeeds on my system, even though that's a ludicrously high number. (It is
>> more than half the number of bytes of memory my computer has.)
>> 
>> 
>> So why can't Python tell if I'm setting the limit too high?
>> 
>> (I'm assuming that if it could, it would.)
>
> I believe the issue here is that Python recursion results in the C stack
> growing.  Each Python function call creates a number of C function calls,
> which grows the C stack.  The crash you see is the C stack overflowing.
>
> Is there a way to know how large the C stack can grow, and how much it
> will grow for each Python function call? That sounds complicated to get
> right.
>
> --Ned.

It's probably trivial to look at a number and say "Yeah, no, that's
CLEARLY too high." based on the minimum number of bytes a stack frame
can require. Guaranteeing that some number lower than that is safe is
almost certainly impossible. So you'd get an exception for truly
stupid numbers, but a lack of exception is no guarantee of safety.
Which is worth what it's worth, I guess.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.