[py-svn] commit/pytest: hpk42: refine documentation, move setup to own "setup" page and provide
Bitbucket
commits-noreply at bitbucket.org
Thu Aug 2 12:08:03 CEST 2012
1 new commit in pytest:
https://bitbucket.org/hpk42/pytest/changeset/e9293dbade35/
changeset: e9293dbade35
user: hpk42
date: 2012-08-02 12:07:54
summary: refine documentation, move setup to own "setup" page and provide
some more examples. move old setup_module/... to xunit_old page.
affected #: 5 files
diff -r d130a4163d9a833f55d4fc913d0e676e869b44bc -r e9293dbade357ca7ac20a796f0f3484ae70b06ab doc/en/index.txt
--- a/doc/en/index.txt
+++ b/doc/en/index.txt
@@ -25,7 +25,9 @@
- **supports functional testing and complex test setups**
- - (new in 2.3) :ref:`easy test resource management and generalized xUnit setup <resources>`
+ - (new in 2.3) :ref:`easy test resource management, scoping and
+ parametrization <resources>`
+ - (new in 2.3) :ref:`xunitsetup`.
- (new in 2.2) :ref:`durations`
- (much improved in 2.2) :ref:`marking and test selection <mark>`
- (improved in 2.2) :ref:`parametrized test functions <parametrized test functions>`
diff -r d130a4163d9a833f55d4fc913d0e676e869b44bc -r e9293dbade357ca7ac20a796f0f3484ae70b06ab doc/en/resources.txt
--- a/doc/en/resources.txt
+++ b/doc/en/resources.txt
@@ -1,74 +1,66 @@
.. _resources:
-test resource management and xUnit setup (on steroids)
+=======================================================
+test resource injection and parametrization
=======================================================
.. _`Dependency injection`: http://en.wikipedia.org/wiki/Dependency_injection
.. versionadded: 2.3
-pytest offers advanced resource parametrization and injection mechanisms
-including a fully integrated generalization of the popular xUnit
-setup-style methods. A resource is created by a ``@pytest.mark.factory``
-marked function and its name is the name of the function. A resource
-is injected into test or setup functions if they use the name
-in their signature. Therefore and also for historic reasons, resources are
-sometimes called "funcargs" because they ultimately appear as
-function arguments.
-
-The pytest resource management and setup features are exposed through
-three decorators:
-
-* a `@pytest.mark.factory`_ marker to define resource factories,
- their scoping and parametrization.
-
-* a `@pytest.mark.setup`_ marker to define setup functions and their
- scoping.
-
-* a `@pytest.mark.parametrize`_ marker for executing test functions
- multiple times with different parameter sets
-
-Generally, resource factories and setup functions:
-
-- can be defined in test modules, test classes, conftest.py files or
- in plugins.
-
-- can themselves receive resources through their function arguments,
- simplifying the setup and use of interdependent resources.
-
-- can use the special `testcontext`_ object for access to the
- context in which the factory/setup is called and for registering
- finalizers.
-
-This document showcases these features through some basic examples.
-
-Note that pytest also comes with some :ref:`builtinresources` which
-you can use without defining them yourself.
-
-Background and terms
----------------------------
+pytest offers very flexible means for managing test resources and
+test parametrization.
The pytest resource management mechanism is an example of `Dependency
-Injection`_ which helps to de-couple test code from resource
-instantiation code required for them to execute. At test writing time
-you typically do not need to care for the details of how your required
-resources are constructed, if they live through a function, class,
-module or session scope or if the test will be called multiple times
-with different resource instances.
+Injection`_ because test and :ref:`setup functions <xunitsetup>` receive
+resources simply by stating them as an input argument. Therefore and
+also for historic reasons, they are often called **funcargs**. At test
+writing time you typically do not need to care for the details of how
+your required resources are constructed, if they live through a
+function, class, module or session scope or if the test will be called
+multiple times with different resource instances.
To create a value with which to call a test function a resource factory
function is called which gets full access to the test context and can
register finalizers which are to be run after the last test in that context
finished. Resource factories can be implemented in same test class or
-test module, in a per-directory ``conftest.py`` file or in an external plugin. This allows total de-coupling of test and setup code.
+test module, in a per-directory ``conftest.py`` file or in an external
+plugin. This allows **total de-coupling of test and setup code**,
+lowering the cost of refactoring.
A test function may be invoked multiple times in which case we
-speak of :ref:`parametrized testing <parametrizing-tests>`. This can be
-very useful if you want to test e.g. against different database backends
-or with multiple numerical arguments sets and want to reuse the same set
-of test functions.
+speak of parametrization. You can parametrize resources or parametrize
+test function arguments directly or even implement your own parametrization
+scheme through a plugin hook.
+A resource has a **name** under which test and setup functions
+can access it by listing it as an input argument. Due to this and
+also for historic reasons, resources are often called **funcargs**.
+A resource is created by a factory which can be flagged with a **scope**
+to only create resources on a per-class/per-module/per-session basis
+instead of the default per-function scope.
+
+Concretely, there are three means of resource and parametrization management:
+
+* a `@pytest.mark.factory`_ marker to define resource factories,
+ their scoping and parametrization. Factories can themselves
+ receive resources through their function arguments, easing
+ the setup of interdependent resources. They can also use
+ the special `testcontext`_ object to access details n which
+ the factory/setup is called and for registering finalizers.
+
+* a `@pytest.mark.parametrize`_ marker for executing test functions
+ multiple times with different parameter sets
+
+* a `pytest_generate_tests`_ plugin hook marker for implementing
+ your parametrization for a test function which may depend on
+ command line options, class/module attributes etc.
+
+Finally, pytest comes with some :ref:`builtinresources` which
+you can use without defining them yourself. Moreover, third-party
+plugins offer their own resources so that after installation
+you can simply use them in your test and setup functions.
.. _`@pytest.mark.factory`:
@@ -81,11 +73,12 @@
The `@pytest.mark.factory`_ marker allows to
-* mark a function as a factory for resources used by test and setup functions
-* define parametrization to run tests multiple times with different
+* mark a function as a factory for resources, useable by test and setup functions
+* define parameters in order to run tests multiple times with different
resource instances
-* set a scope which determines the level of caching. valid scopes
- are ``session``, ``module``, ``class`` and ``function``.
+* set a scope which determines the level of caching, i.e. how often
+ the factory will be called. Valid scopes are ``session``, ``module``,
+ ``class`` and ``function``.
Here is a simple example of a factory creating a shared ``smtplib.SMTP``
connection resource which test functions then may use across the whole
@@ -248,12 +241,12 @@
the number of active resources at any given time.
-Accessing resources from a factory function
+Interdepdendent resources
----------------------------------------------------------
-You can directly use resources as funcargs in resource factories.
-Extending the previous example we can instantiate an application
-object and stick the live ``smtp`` resource into it::
+You can not only use resources in test functions but also in resource factories
+themselves. Extending the previous example we can instantiate an application
+object by sticking the ``smtp`` resource into it::
# content of test_appsetup.py
@@ -289,114 +282,6 @@
There is no need for the ``app`` factory to be aware of the parametrization.
-.. _`new_setup`:
-.. _`@pytest.mark.setup`:
-
-``@pytest.mark.setup``: xUnit setup methods on steroids
------------------------------------------------------------------
-
-.. regendoc:wipe
-
-.. versionadded:: 2.3
-
-The ``@pytest.mark.setup`` marker allows
-
-* to define setup-functions close to test code or in conftest.py files
- or plugins.
-* to mark a function as a setup method; the function can itself
- receive funcargs and will execute multiple times if the funcargs
- are parametrized
-* to set a scope which influences when the setup function going to be
- called. valid scopes are ``session``, ``module``, ``class`` and ``function``.
-
-Here is a simple example. First we define a global ``globdir`` resource::
-
- # content of conftest.py
- import pytest
-
- @pytest.mark.factory(scope="module")
- def globdir(testcontext, tmpdir):
- def fin():
- print "finalize", tmpdir
- testcontext.addfinalizer(fin)
- print "created resource", tmpdir
- return tmpdir
-
-And then we write a test file containing a setup-marked function
-taking this resource and setting it as a module global::
-
- # content of test_module.py
- import pytest
-
- @pytest.mark.setup(scope="module")
- def setresource(testcontext, globdir):
- print "setupresource", globdir
- testcontext.module.myresource = globdir
-
- def test_1():
- assert myresource
- print "using myresource", myresource
-
- def test_2():
- assert myresource
- print "using myresource", myresource
-
-Let's run this module::
-
- $ py.test -qs
- collecting ... collected 2 items
- ..
- 2 passed in 0.26 seconds
- created resource /home/hpk/tmp/pytest-4427/test_10
- setupresource /home/hpk/tmp/pytest-4427/test_10
- using myresource /home/hpk/tmp/pytest-4427/test_10
- using myresource /home/hpk/tmp/pytest-4427/test_10
- finalize /home/hpk/tmp/pytest-4427/test_10
-
-The two test functions in the module use the same global ``myresource``
-object because the ``setresource`` set it as a module attribute.
-
-The ``globdir`` factory can now become parametrized without any test
-or setup code needing to change::
-
- # content of conftest.py
- import pytest
-
- @pytest.mark.factory(scope="module", params=["aaa", "bbb"])
- def globdir(testcontext, tmpdir):
- newtmp = tmpdir.join(testcontext.param)
- def fin():
- print "finalize", newtmp
- testcontext.addfinalizer(fin)
- print "created resource", newtmp
- return newtmp
-
-Running the unchanged previous test files now runs four tests::
-
- $ py.test -qs
- collecting ... collected 4 items
- ....
- 4 passed in 0.26 seconds
- created resource /home/hpk/tmp/pytest-4428/test_1_aaa_0/aaa
- setupresource /home/hpk/tmp/pytest-4428/test_1_aaa_0/aaa
- using myresource /home/hpk/tmp/pytest-4428/test_1_aaa_0/aaa
- using myresource /home/hpk/tmp/pytest-4428/test_1_aaa_0/aaa
- finalize /home/hpk/tmp/pytest-4428/test_1_aaa_0/aaa
- created resource /home/hpk/tmp/pytest-4428/test_1_bbb_0/bbb
- setupresource /home/hpk/tmp/pytest-4428/test_1_bbb_0/bbb
- using myresource /home/hpk/tmp/pytest-4428/test_1_bbb_0/bbb
- using myresource /home/hpk/tmp/pytest-4428/test_1_bbb_0/bbb
- finalize /home/hpk/tmp/pytest-4428/test_1_bbb_0/bbb
-
-Each parameter causes the creation of a respective resource and the
-unchanged test module uses it in its ``@setup`` decorated method.
-
-.. note::
-
- Tests using a particular parametrized resource instance will
- executed next to each other. Any finalizers will be run before the
- next parametrized resource instance is being setup and tests
- are rerun.
Grouping tests by resource parameters
----------------------------------------------------------
@@ -540,7 +425,10 @@
Note that there are various ways how you can mark groups of functions,
see :ref:`mark`.
-Generating parameters combinations, depending on command line
+
+.. _`pytest_generate_tests`:
+
+``pytest_generate_test``: implementing your own parametrization scheme
----------------------------------------------------------------------------
.. regendoc:wipe
diff -r d130a4163d9a833f55d4fc913d0e676e869b44bc -r e9293dbade357ca7ac20a796f0f3484ae70b06ab doc/en/setup.txt
--- /dev/null
+++ b/doc/en/setup.txt
@@ -0,0 +1,204 @@
+.. _xunitsetup:
+.. _setup:
+
+``@setup`` functions or: xunit on steroids
+========================================================
+
+.. versionadded:: 2.3
+
+.. _`funcargs`: funcargs.html
+.. _`test parametrization`: funcargs.html#parametrizing-tests
+.. _`unittest plugin`: plugin/unittest.html
+.. _`xUnit`: http://en.wikipedia.org/wiki/XUnit
+
+Python, Java and many other languages support a so called xUnit_ style
+of resource setup. This typically involves the call of a ``setup``
+("fixture") method before running a test function and ``teardown`` after
+it has finished. Unlike :ref:`injected resources <resources>` setup
+functions work indirectly by causing global side effects or
+setting test case attributes which test methods can then access.
+
+pytest originally introduced in 2005 a fine-grained model of detecting
+setup and teardown functions on a per-module, class or function basis.
+The Python unittest module and nose have subsequently incorporated them.
+This model remains supported as :ref:`old-style xunit`.
+
+With pytest-2.3 a new ``pytest.setup()`` decorator is introduced
+to mark functions as setup functions which:
+
+- can receive resources through funcargs,
+- fully interoperate with parametrized resources,
+- can be defined in a plugin or conftest.py file and get called
+ on a per-session, per-module, per-class or per-function basis,
+- can access the full :ref:`testcontext` for which the setup is called,
+- can precisely control teardown by registering one or multiple
+ teardown functions as soon as they have performed some actions
+ which need undoing, eliminating the no need for a separate
+ "teardown" decorator.
+- allow to separate different setup concerns even if they
+ happen to work in the same scope
+
+All of these features are now demonstrated by little examples.
+
+.. _`new_setup`:
+.. _`@pytest.mark.setup`:
+
+basic per-function setup
+-------------------------------
+
+.. regendoc:wipe
+
+Suppose you want to have a clean directory with a single
+file entry for each test function in a module and have
+the test execute with this directory as current working dir::
+
+ # content of test_funcdir.py
+ import pytest
+ import os
+
+ @pytest.mark.setup()
+ def mydir(tmpdir):
+ tmpdir.join("myfile").write("example content")
+ old = tmpdir.chdir()
+
+ def test_function1():
+ assert os.path.exists("myfile")
+ f = open("anotherfile", "w")
+ f.write("")
+ f.close()
+
+ def test_function2():
+ assert os.path.exists("myfile")
+ assert not os.path.exists("anotherfile")
+
+Our ``mydir`` setup function is executed on a per-function basis,
+the default scope used by the ``pytest.mark.setup`` decorator.
+It accesses the ``tmpdir`` resource which provides a new empty
+directory path object. The ``test_function2`` here checks that
+it executes with a fresh directory and specifically
+does not see the previously created ``anotherfile``. We can
+thus expect two passing tests::
+
+ $ py.test -v
+ =========================== test session starts ============================
+ platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev7 -- /home/hpk/venv/1/bin/python
+ cachedir: /home/hpk/tmp/doc-exec-410/.cache
+ plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
+ collecting ... collected 2 items
+
+ test_funcdir.py:9: test_function1 PASSED
+ test_funcdir.py:15: test_function2 PASSED
+
+ ========================= 2 passed in 0.26 seconds =========================
+
+per-function setup, for every function of a project
+------------------------------------------------------------
+
+If you want to define a setup per-function but want to apply
+it to every function in your project you don't need to duplicate
+the setup-definition into each test module. Instead you can put
+it into a ``conftest.py`` file into the root of your project::
+
+ # content of conftest.py
+ import pytest
+ import os
+
+ @pytest.mark.setup()
+ def cleandir(tmpdir):
+ old = tmpdir.chdir()
+
+The ``cleandir`` setup function will be called for every test function
+below the directory tree where ``conftest.py`` resides. In this
+case it just uses the builtin ``tmpdir`` resource to change to the
+empty directory ahead of running a test.
+
+test modules accessing a global resource
+-------------------------------------------------------
+
+If you want test modules to access a global resource,
+you can stick the resource to the module globals in
+a per-module setup function. We use a :ref:`resource factory
+<@pytest.mark.factory>` to create our global resource::
+
+ # content of conftest.py
+ import pytest
+
+ class GlobalResource:
+ def __init__(self):
+ pass
+
+ @pytest.mark.factory(scope="session")
+ def globresource():
+ return GlobalResource()
+
+ @pytest.mark.setup(scope="module")
+ def setresource(testcontext, globresource):
+ testcontext.module.globresource = globresource
+
+Now any test module can access ``globresource`` as a module global::
+
+ # content of test_glob.py
+
+ def test_1():
+ print ("test_1 %s" % globresource)
+ def test_2():
+ print ("test_2 %s" % globresource)
+
+Let's run this module without output-capturing::
+
+ $ py.test -qs test_glob.py
+ collecting ... collected 2 items
+ ..
+ 2 passed in 0.02 seconds
+ test_1 <conftest.GlobalResource instance at 0x13197e8>
+ test_2 <conftest.GlobalResource instance at 0x13197e8>
+
+The two tests see the same global ``globresource`` object.
+
+Parametrizing the global resource
++++++++++++++++++++++++++++++++++++++++++++++++++
+
+We extend the previous example and add parametrization to the globresource
+factory and also add a finalizer::
+
+ # content of conftest.py
+
+ import pytest
+
+ class GlobalResource:
+ def __init__(self, param):
+ self.param = param
+
+ @pytest.mark.factory(scope="session", params=[1,2])
+ def globresource(testcontext):
+ g = GlobalResource(testcontext.param)
+ def fin():
+ print "finalizing", g
+ testcontext.addfinalizer(fin)
+ return g
+
+ @pytest.mark.setup(scope="module")
+ def setresource(testcontext, globresource):
+ testcontext.module.globresource = globresource
+
+And then re-run our test module::
+
+ $ py.test -qs test_glob.py
+ collecting ... collected 4 items
+ ....
+ 4 passed in 0.02 seconds
+ test_1 <conftest.GlobalResource instance at 0x1922e18>
+ test_2 <conftest.GlobalResource instance at 0x1922e18>
+ finalizing <conftest.GlobalResource instance at 0x1922e18>
+ test_1 <conftest.GlobalResource instance at 0x1925518>
+ test_2 <conftest.GlobalResource instance at 0x1925518>
+ finalizing <conftest.GlobalResource instance at 0x1925518>
+
+We are now running the two tests twice with two different global resource
+instances. Note that the tests are ordered such that only
+one instance is active at any given time: the finalizer of
+the first globresource instance is called before the second
+instance is created and sent to the setup functions.
+
+
+
diff -r d130a4163d9a833f55d4fc913d0e676e869b44bc -r e9293dbade357ca7ac20a796f0f3484ae70b06ab doc/en/xunit_old.txt
--- /dev/null
+++ b/doc/en/xunit_old.txt
@@ -0,0 +1,76 @@
+
+.. _`old-style xunit`:
+
+Old-style xunit-style setup
+========================================
+
+This section describes the old way how you can implement setup and
+teardown on a per-module/class/function basis. It remains fully
+supported but it is recommended to rather use :ref:`@setup functions <setup>`
+or :ref:`injected resources <resources>` for implementing your setup needs.
+
+Module level setup/teardown
+--------------------------------------
+
+If you have multiple test functions and test classes in a single
+module you can optionally implement the following fixture methods
+which will usually be called once for all the functions::
+
+ def setup_module(module):
+ """ setup any state specific to the execution of the given module."""
+
+ def teardown_module(module):
+ """ teardown any state that was previously setup with a setup_module
+ method.
+ """
+
+Class level setup/teardown
+----------------------------------
+
+Similarly, the following methods are called at class level before
+and after all test methods of the class are called::
+
+ @classmethod
+ def setup_class(cls):
+ """ setup any state specific to the execution of the given class (which
+ usually contains tests).
+ """
+
+ @classmethod
+ def teardown_class(cls):
+ """ teardown any state that was previously setup with a call to
+ setup_class.
+ """
+
+Method and function level setup/teardown
+-----------------------------------------------
+
+Similarly, the following methods are called around each method invocation::
+
+ def setup_method(self, method):
+ """ setup any state tied to the execution of the given method in a
+ class. setup_method is invoked for every test method of a class.
+ """
+
+ def teardown_method(self, method):
+ """ teardown any state that was previously setup with a setup_method
+ call.
+ """
+
+If you would rather define test functions directly at module level
+you can also use the following functions to implement fixtures::
+
+ def setup_function(function):
+ """ setup any state tied to the execution of the given function.
+ Invoked for every test function in the module.
+ """
+
+ def teardown_function(function):
+ """ teardown any state that was previously setup with a setup_function
+ call.
+ """
+
+Note that it is possible for setup/teardown pairs to be invoked multiple times
+per testing process.
+
+.. _`unittest.py module`: http://docs.python.org/library/unittest.html
diff -r d130a4163d9a833f55d4fc913d0e676e869b44bc -r e9293dbade357ca7ac20a796f0f3484ae70b06ab doc/en/xunit_setup.txt
--- a/doc/en/xunit_setup.txt
+++ /dev/null
@@ -1,83 +0,0 @@
-.. _xunitsetup:
-
-====================================
-Extended xUnit style setup fixtures
-====================================
-
-.. _`funcargs`: funcargs.html
-.. _`test parametrization`: funcargs.html#parametrizing-tests
-.. _`unittest plugin`: plugin/unittest.html
-.. _`xUnit`: http://en.wikipedia.org/wiki/XUnit
-
-Python, Java and many other languages support xUnit_ style testing.
-This typically involves the call of a ``setup`` ("fixture") method
-before running a test function and ``teardown`` after it has finished.
-``py.test`` supports a more fine-grained model of setup/teardown
-handling by optionally calling per-module and per-class hooks.
-
-
-Module level setup/teardown
-=============================================
-
-If you have multiple test functions and test classes in a single
-module you can optionally implement the following fixture methods
-which will usually be called once for all the functions::
-
- def setup_module(module):
- """ setup any state specific to the execution of the given module."""
-
- def teardown_module(module):
- """ teardown any state that was previously setup with a setup_module
- method.
- """
-
-Class level setup/teardown
-=============================================
-
-Similarly, the following methods are called at class level before
-and after all test methods of the class are called::
-
- @classmethod
- def setup_class(cls):
- """ setup any state specific to the execution of the given class (which
- usually contains tests).
- """
-
- @classmethod
- def teardown_class(cls):
- """ teardown any state that was previously setup with a call to
- setup_class.
- """
-
-Method and function level setup/teardown
-=============================================
-
-Similarly, the following methods are called around each method invocation::
-
- def setup_method(self, method):
- """ setup any state tied to the execution of the given method in a
- class. setup_method is invoked for every test method of a class.
- """
-
- def teardown_method(self, method):
- """ teardown any state that was previously setup with a setup_method
- call.
- """
-
-If you would rather define test functions directly at module level
-you can also use the following functions to implement fixtures::
-
- def setup_function(function):
- """ setup any state tied to the execution of the given function.
- Invoked for every test function in the module.
- """
-
- def teardown_function(function):
- """ teardown any state that was previously setup with a setup_function
- call.
- """
-
-Note that it is possible for setup/teardown pairs to be invoked multiple times
-per testing process.
-
-.. _`unittest.py module`: http://docs.python.org/library/unittest.html
Repository URL: https://bitbucket.org/hpk42/pytest/
--
This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.
More information about the pytest-commit
mailing list