[Python-Dev] incorrect regression tests

[Tim Peters]

[Neal Norwitz]
> I found 7 regression tests which weren't being executed by regrtest.
> I checked in fixes for 6 of them.

Good catch!

> test_zipimport was the other test, but there was a problem.  More on
> that below.

Ditto.

> To be run, tests must do one of following:
>
>         1) have a def test_main() which runs the tests
>         2) run the tests as a side-effect of import

New tests should do #1 (IMO).  #2 means the import lock is held for the
duration of the test, and that can create deadlock problems (indeed, I added
the test_main() business to make it possible to run some thread tests
reliably -- besides, it's always a mental strain when "big things" happen as
a side effect).

> I'm wondering if regrtest.py should grow an option to print a list of
> all the tests which don't define test_main?  When adding a new test,
> it will be easy to verify the test is, in fact, being run.  Perhaps
> also a list of known tests which don't have a test_main and are
> expected to run on import? (I think there's over 100.)

I'm not sure how this would help unless combined with a crusade to enforce
that all tests have a test_main.  Then regrtest could simply complain when
there isn't one.

> When test_zipimport is run by regrtest, there is memory corruption:
>
> [neal@epoch 2.3]$ ./python -E -tt ./Lib/test/regrtest.py test_zipimport
> test_zipimport
> Debug memory block at address p=0x40280508:
>     74 bytes originally requested

That's an unusual # of bytes requested, because it's not divisible by 4.
Best guess is that it's string memory.

>     The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected.
>     The 4 pad bytes at tail=0x40280552 are not all FORBIDDENBYTE (0xfb):
>         at tail+0: 0x00 *** OUCH
>         at tail+1: 0xfb
>         at tail+2: 0xfb
>         at tail+3: 0xfb

So code wrote \x00 into a single byte following the requested area:  best
guess now is that it's string memory and that whoever asked for it forgot to
ask for room for a trailing 0 byte.

>     The block was made by call #51959 to debug malloc/realloc.

In obmalloc.c:

"""
/* serialno is always incremented via calling this routine.  The point is
   to supply a single place to set a breakpoint.
*/
static void
bumpserialno(void)
{
	++serialno;
}
"""

If the test is deterministic, you can set a conditional breakpoint there for
when serialno reaches 51959, and catch the offending malloc/realloc call in
the act.

>     Data at p: 64 65 66 20 67 65 74 5f ... 69 6c 65 5f 5f 0a 00 0a
                  d  e  f     g  e  t  _ ...  i  l  e  _  _ \n \0 \n

It's probably relevant that test_zipimport.py contains

        def get_file():
            return __file__

in a string passed to compile() (and perhaps that Guido recently changed
compile()).