make test segfaults with "--enable-shared" on Python 2.3.3

[Andrew MacIntyre]

On Mon, 29 Dec 2003, Berthold Höllmann wrote:

> so it seems that the other modules loaded in the "make test" run
> decrease the avaliable stack so that the test does not succeed but
> calling the test alone finds enough stack space avaliable?

The above message crossed with my other reply.

This looks very much like the stack problem I referred to, which I am
familiar with on FreeBSD.  I have been able to build with
--enable-shared on FreeBSD (after some configure tweaking), and see a
similar effect - ie test_re coredumps with --enable-shared and passes
without (and this with gcc 2.95.4).  From the backtrace, I see that a
USE_RECURSION_LIMIT of 7200 would have worked, compared to the 7500 that
works for the standard build.

That it passes the individual test but not the full test suite for you
suggests that your build is right on the stacksize limit, and lowering
USE_RECURSION_LIMIT by only a small amount would get you through.

Note that there are some subtleties here - even though your ulimit says
unlimited stackspace, in the presence of threads this may not hold.
It certainly doesn't on FreeBSD 4.x where there the pthreads support has a
hardcoded 1MB stack for the primary thread (which is what is running the
regression test and is distinct from the stack created for each
instantiated thread).  The LinuxThreads port on FreeBSD seems not to have
the same restriction.  Exactly what is happening on your Linux system I
don't know; you would have to do some research to find out.

Depending on how you wish to proceed, you have a couple of options:
- build normally, then blow away Modules/_sre.o and recompile
  Modules/_sre.c with -Os (just temporarily doctor the Makefile).
  From what I've seen -Os cuts the stack consumption enough to get
  you through the full test suite, and doesn't seriously impact sre's
  performance.
- modify the USE_RECURSION_LIMIT macro in Modules/_sre.c to a lower value
  and rebuild. Linux would be using the default definition of 10000 at
  line 98 (for 2.3.3).
- find a way to change the default stack size for the primary thread.

If you play with the second or third options, a report to the Python bug
tracker on SF with the conclusion you reach would be useful.  Note that
such a bug report should include details of OS & compiler, so that
suitable #ifdef'ery can be written, and should be based on Python's
default compiler switches.  As I noted previously, sre in Python 2.4 will
not have this issue, but Anthony Baxter has said that he expects to do a
2.3.4 release (April/May O4 as I recall).

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac at pcug.org.au             (alt) |        Belconnen  ACT  2616
Web:    http://www.andymac.org/               |        Australia