[Pytest-commit] Issue #658: Handling of SystemExit if different pid (after os.fork) (hpk42/pytest)

[Torsten Landschoff]

New issue 658: Handling of SystemExit if different pid (after os.fork)
https://bitbucket.org/hpk42/pytest/issue/658/handling-of-systemexit-if-different-pid

Torsten Landschoff:

Hi *,

I am trying to use our self written ForkServer (something like https://hg.python.org/cpython/file/c917ba25c007/Lib/multiprocessing/forkserver.py but for python2) in unit tests with py.test.

## Background ##

It turned out that using os._exit in each child process forked by the ForkServer is not a good idea, because each child process assumes to be a real python process with atexit.register working and correct cleanup of global resources. This causes a deviation between Windows and Linux as well, as we use multiprocessing to start child processes on Windows where atexit and friends work fine of course.

Now the ForkServer is basically a [zygote process](https://code.google.com/p/chromium/wiki/LinuxZygote) and is started really early in our application setup code. Therefore we can switch to using sys.exit() just fine, nothing will catch the SystemExit exception from unwinding.

## Problem ##

Now this change kills a lot of unit tests because the SystemExit exception propagates to py.test when raised from a fork server created inside a unit test. Obviously using sys.exit here is probably not a good idea, because this will run cleanups from a lot of stuff (other tests, pytest itself etc.).

So what I would like to do is to wrap each test function and catch SystemExit there. If SystemExit comes from a different process id that the original pid, this should be mapped to os._exit(), otherwise the SystemExit exception should be propagated as usual.

I think this would even be the correct course of action in pytest proper.

Unfortunately I can not for the life of me figure out how to do this. I tried to implement the hook pytest_runtest_call in my conftest.py but this only causes the an additional execution of item.runtest() instead of replacing the original implementation.

## Example Code ##

The attached test in test_fork.py illustrates the problem. Running it I get the following output:


```
(env)torsten at horatio:~/pytest-fork/bug-fork$ py.test -vrsx
============================= test session starts ==============================
platform linux2 -- Python 2.7.3 -- py-1.4.26 -- pytest-2.7.0.dev1 -- /home/torsten/pytest-fork/env/bin/python
collected 1 items 

test_fork.py::test_can_test_fork PASSED

=========================== 1 passed in 0.01 seconds ===========================


=================================== FAILURES ===================================
______________________________ test_can_test_fork ______________________________

    @pytest.mark.skipif(not hasattr(os, "fork"),
            reason="os.fork is missing, I need a real operating system")
    def test_can_test_fork():
        pid = os.fork()
        if not pid:
>           sys.exit()
E           SystemExit

test_fork.py:9: SystemExit
=========================== 1 failed in 0.02 seconds ===========================
(env)torsten at horatio:~/pytest-fork/bug-fork$ 
```

What I would have expected is successful termination.

## Attempted patch ##

The attachment catch_system_exit.diff contains a patch that fixes the problem for me. Your mileage may vary, applying this everywhere might not be such a good idea.

Unfortunately I can not easily replace pytest on all our build machines with a custom build. I'd really like to stay with a released version and add this behaviour via conftest. Hints greatly appreciated.