How to avoid MemoryError on recursion?

[Martin von Loewis]

Dave.Haynes at sia-mce.co.uk (Dave Haynes) writes:

>  I'm using John Aycock's SPARK (
> http://www.csr.uvic.ca/~aycock/python/ ) to process some files. Part
> of the SPARK semantic checker uses a recursive function to traverse
> the syntax tree. I've had to increase the recursion limit to 10000 to
> avoid RuntimeError, and I now get 'MemoryError: Stack overflow'.
>  
>  Is there anything I can do to get Python more allocated memory (or
> get it to use less)? I'm using Python 2.0 on a Windows NT4 box with
> 256 Mbyte RAM. Task Manager registers only 26% of this in use while
> running the script.

If you have exceeded the stack limit, it helps little that there is
still memory left: the stack limit is much less than the available
memory. You can, on Windows, increase the stack limit by changing a
value in hte executable (or passing that value to the linker when
linking the executable); for that to work, you need to relink Python.

One option you may consider is to use stackless Python. I don't know
whether SPARK uses the C stack on its own, or whether it does the
recursion in Python. If it is the latter, stackless Python should give
you much deeper recursion nesting.

>  Anyone more familiar with SPARK than I am might be able to give some
> tips on how to flatten the AST tree. I'm working from a fixed BNF
> description, which has plenty of hierarchy in it.

Sorry, can't help here. I recommend to contact John directly; he may
have suggestions.

Regards,
Martin