[Tutor] PyDoc problem

[Kent Johnson]

Christian Meesters wrote:
> Hi,
> 
> PyDoc is working well, if invoked without flags on the modul in  
> question. But when I use it with '-w' to produce html-output, I get an  
> extremely long Traceback after a runtime of an hour or two. Here is a  
> short part from it:
> "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
> python2.3/sre.py", line 137, in search
>       return _compile(pattern, flags).search(string)
>   RuntimeError: maximum recursion depth exceeded
> 
> What's going wrong? When I ask Pydoc to write html documentation for  
> other modules there is no problem at all. It just works fine. And this  
> traceback tell me nothing about my module, only about PyDoc. For it  
> matters: I'm stuck with Python2.3 for compatibility reasons in this  
> project and the module I'd like to see documented uses scipy and it is  
> not really a small module anymore (about 1000 lines, including  
> docstrings and comments).

Some re searches in Python 2.3 can cause stack overflow. It looks like you may be running in to that problem. It is fixed in Python 2.4; could you use 2.4 to run pydoc?

Here is the relevant section of the Python 2.4 release notes:
The re module is also no longer recursive, thanks to a massive amount of work by Gustavo Niemeyer. In a recursive regular expression engine, certain patterns result in a large amount of C stack space being consumed, and it was possible to overflow the stack. For example, if you matched a 30000-byte string of "a" characters against the expression (a|b)+, one stack frame was consumed per character. Python 2.3 tried to check for stack overflow and raise a RuntimeError exception, but certain patterns could sidestep the checking and if you were unlucky Python could segfault. Python 2.4's regular expression engine can match this pattern without problems.

Kent