[Python-Dev] Fwd: Allowing to run certain regression tests in subprocesses

[Eli Bendersky]

On Sat, Aug 3, 2013 at 6:57 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Sat, 03 Aug 2013 16:47:37 -0700, Eli Bendersky <eliben at gmail.com> wrote:
>> On Sat, Aug 3, 2013 at 4:36 PM, Eli Bendersky <eliben at gmail.com> wrote:
>> > Hi All,
>> >
>> > Today the issue of cross-test global env dependencies showed its ugly
>> > head again for me. I recall a previous discussion
>> > (http://mail.python.org/pipermail/python-dev/2013-January/123409.html)
>> > but there were many more over the years.
>> >
>> > The core problem is that some tests modify the global env
>> > (particularly importing modules) and this sometimes has adverse
>> > effects on other tests, because test.regrtest runs all tests in a
>> > single process. In the discussion linked above, the particular culprit
>> > test__all__ was judged as a candidate to be moved to a subprocess.
>> >
>> > I want to propose adding a capability to our test harness to run
>> > specific tests in subprocesses. Each test will have some simple way of
>> > asking to be run in a subprocess, and regrtest will concur (even when
>> > running -j1). test__all__ can go there, and it can help solve other
>> > problems.
>> >
>> > My particular case is trying to write a test for
>> > http://bugs.python.org/issue14988 - wherein I have to simulate a
>> > situation of non-existent pyexpat. It's not hard to write a test for
>> > it, but when run in tandem with other tests (where C extensions loaded
>> > pyexpat) it becomes seemingly impossible to set up. This should not be
>> > the case - there's nothing wrong with wanting to simulate this case,
>> > and there's nothing wrong in Python and the stdlib - it's purely an
>> > artifact of the way our regression suite works.
>> >
>> > Thoughts?
>> >
>> > Eli
>>
>> FWIW the problem is also discussed here:
>> http://bugs.python.org/issue1674555, w.r.t. test_site
>
> Can't you just launch a subprocess from the test itself using script_helpers?
>

[sorry, sent privately by mistake; forwarding to pydev]

I can, but such launching will be necessarily duplicated across all
tests that need this functionality (test_site, test___all__, etc).
Since regrtest already has functionality for launching whole
test-suites in subprocesses, it makes sense to reuse it, no?

Eli