[Pytest-commit] Issue #658: Handling of SystemExit if different pid (after os.fork) (hpk42/pytest)
Torsten Landschoff
issues-reply at bitbucket.org
Wed Jan 14 10:36:12 CET 2015
New issue 658: Handling of SystemExit if different pid (after os.fork)
https://bitbucket.org/hpk42/pytest/issue/658/handling-of-systemexit-if-different-pid
Torsten Landschoff:
Hi *,
I am trying to use our self written ForkServer (something like https://hg.python.org/cpython/file/c917ba25c007/Lib/multiprocessing/forkserver.py but for python2) in unit tests with py.test.
## Background ##
It turned out that using os._exit in each child process forked by the ForkServer is not a good idea, because each child process assumes to be a real python process with atexit.register working and correct cleanup of global resources. This causes a deviation between Windows and Linux as well, as we use multiprocessing to start child processes on Windows where atexit and friends work fine of course.
Now the ForkServer is basically a [zygote process](https://code.google.com/p/chromium/wiki/LinuxZygote) and is started really early in our application setup code. Therefore we can switch to using sys.exit() just fine, nothing will catch the SystemExit exception from unwinding.
## Problem ##
Now this change kills a lot of unit tests because the SystemExit exception propagates to py.test when raised from a fork server created inside a unit test. Obviously using sys.exit here is probably not a good idea, because this will run cleanups from a lot of stuff (other tests, pytest itself etc.).
So what I would like to do is to wrap each test function and catch SystemExit there. If SystemExit comes from a different process id that the original pid, this should be mapped to os._exit(), otherwise the SystemExit exception should be propagated as usual.
I think this would even be the correct course of action in pytest proper.
Unfortunately I can not for the life of me figure out how to do this. I tried to implement the hook pytest_runtest_call in my conftest.py but this only causes the an additional execution of item.runtest() instead of replacing the original implementation.
## Example Code ##
The attached test in test_fork.py illustrates the problem. Running it I get the following output:
```
(env)torsten at horatio:~/pytest-fork/bug-fork$ py.test -vrsx
============================= test session starts ==============================
platform linux2 -- Python 2.7.3 -- py-1.4.26 -- pytest-2.7.0.dev1 -- /home/torsten/pytest-fork/env/bin/python
collected 1 items
test_fork.py::test_can_test_fork PASSED
=========================== 1 passed in 0.01 seconds ===========================
=================================== FAILURES ===================================
______________________________ test_can_test_fork ______________________________
@pytest.mark.skipif(not hasattr(os, "fork"),
reason="os.fork is missing, I need a real operating system")
def test_can_test_fork():
pid = os.fork()
if not pid:
> sys.exit()
E SystemExit
test_fork.py:9: SystemExit
=========================== 1 failed in 0.02 seconds ===========================
(env)torsten at horatio:~/pytest-fork/bug-fork$
```
What I would have expected is successful termination.
## Attempted patch ##
The attachment catch_system_exit.diff contains a patch that fixes the problem for me. Your mileage may vary, applying this everywhere might not be such a good idea.
Unfortunately I can not easily replace pytest on all our build machines with a custom build. I'd really like to stay with a released version and add this behaviour via conftest. Hints greatly appreciated.
More information about the pytest-commit
mailing list