[py-svn] pytest-xdist commit 917353071622: shift documentation, remove duplicate code
commits-noreply at bitbucket.org
commits-noreply at bitbucket.org
Thu Nov 18 14:58:20 CET 2010
# HG changeset patch -- Bitbucket.org
# Project pytest-xdist
# URL http://bitbucket.org/hpk42/pytest-xdist/overview
# User holger krekel <holger at merlinux.eu>
# Date 1290088675 -3600
# Node ID 917353071622e40941b1b1d290bdf48f848c8e7a
# Parent 8e7b9712d0dbe19a7577ef706293f9d2e24e24b0
shift documentation, remove duplicate code
--- a/xdist/plugin.py
+++ b/xdist/plugin.py
@@ -1,146 +1,3 @@
-"""loop on failing tests, distribute test runs to CPUs and hosts.
-
-The `pytest-xdist`_ plugin extends py.test with some unique
-test execution modes:
-
-* Looponfail: run your tests repeatedly in a subprocess. After each run py.test
- waits until a file in your project changes and then re-runs the previously
- failing tests. This is repeated until all tests pass after which again
- a full run is performed.
-
-* Load-balancing: if you have multiple CPUs or hosts you can use
- those for a combined test run. This allows to speed up
- development or to use special resources of remote machines.
-
-* Multi-Platform coverage: you can specify different Python interpreters
- or different platforms and run tests in parallel on all of them.
-
-Before running tests remotely, ``py.test`` efficiently synchronizes your
-program source code to the remote place. All test results
-are reported back and displayed to your local test session.
-You may specify different Python versions and interpreters.
-
-.. _`pytest-xdist`: http://pypi.python.org/pypi/pytest-xdist
-
-Usage examples
----------------------
-
-Speed up test runs by sending tests to multiple CPUs
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-To send tests to multiple CPUs, type::
-
- py.test -n NUM
-
-Especially for longer running tests or tests requiring
-a lot of IO this can lead to considerable speed ups.
-
-
-Running tests in a Python subprocess
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-To instantiate a python2.4 sub process and send tests to it, you may type::
-
- py.test -d --tx popen//python=python2.4
-
-This will start a subprocess which is run with the "python2.4"
-Python interpreter, found in your system binary lookup path.
-
-If you prefix the --tx option value like this::
-
- --tx 3*popen//python=python2.4
-
-then three subprocesses would be created and tests
-will be load-balanced across these three processes.
-
-
-Sending tests to remote SSH accounts
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-Suppose you have a package ``mypkg`` which contains some
-tests that you can successfully run locally. And you
-have a ssh-reachable machine ``myhost``. Then
-you can ad-hoc distribute your tests by typing::
-
- py.test -d --tx ssh=myhostpopen --rsyncdir mypkg mypkg
-
-This will synchronize your ``mypkg`` package directory
-to an remote ssh account and then locally collect tests
-and send them to remote places for execution.
-
-You can specify multiple ``--rsyncdir`` directories
-to be sent to the remote side.
-
-**NOTE:** For py.test to collect and send tests correctly
-you not only need to make sure all code and tests
-directories are rsynced, but that any test (sub) directory
-also has an ``__init__.py`` file because internally
-py.test references tests as a fully qualified python
-module path. **You will otherwise get strange errors**
-during setup of the remote side.
-
-Sending tests to remote Socket Servers
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-Download the single-module `socketserver.py`_ Python program
-and run it like this::
-
- python socketserver.py
-
-It will tell you that it starts listening on the default
-port. You can now on your home machine specify this
-new socket host with something like this::
-
- py.test -d --tx socket=192.168.1.102:8888 --rsyncdir mypkg mypkg
-
-
-.. _`atonce`:
-
-Running tests on many platforms at once
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-The basic command to run tests on multiple platforms is::
-
- py.test --dist=each --tx=spec1 --tx=spec2
-
-If you specify a windows host, an OSX host and a Linux
-environment this command will send each tests to all
-platforms - and report back failures from all platforms
-at once. The specifications strings use the `xspec syntax`_.
-
-.. _`xspec syntax`: http://codespeak.net/execnet/trunk/basics.html#xspec
-
-.. _`socketserver.py`: http://codespeak.net/svn/py/dist/py/execnet/script/socketserver.py
-
-.. _`execnet`: http://codespeak.net/execnet
-
-Specifying test exec environments in a conftest.py
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-Instead of specifying command line options, you can
-put options values in a ``conftest.py`` file like this::
-
- option_tx = ['ssh=myhost//python=python2.5', 'popen//python=python2.5']
- option_dist = True
-
-Any commandline ``--tx`` specifictions will add to the list of
-available execution environments.
-
-Specifying "rsync" dirs in a setup.cfg|tox.ini
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-In a ``tox.ini`` or ``setup.cfg`` file in your root project directory
-you may specify directories to include or to exclude in synchronisation::
-
- [pytest]
- rsyncdirs = . mypkg helperpkg
- rsyncignore = .hg
-
-These directory specifications are relative to the directory
-where the configuration file was found.
-
-"""
-
import sys
import py, pytest
--- a/xdist/__init__.py
+++ b/xdist/__init__.py
@@ -1,2 +1,2 @@
#
-__version__ = '1.5a8'
+__version__ = '1.5a9'
--- a/xdist/gwmanage.py
+++ /dev/null
@@ -1,99 +0,0 @@
-"""
- instantiating, managing and rsyncing to test hosts
-"""
-
-import py
-import sys, os.path
-import execnet
-from execnet.gateway_base import RemoteError
-
-class GatewayManager:
- EXIT_TIMEOUT = 10
- RemoteError = RemoteError
- def __init__(self, specs, hook, defaultchdir="pyexecnetcache"):
- self.specs = []
- self.hook = hook
- self.group = execnet.Group()
- for spec in specs:
- if not isinstance(spec, execnet.XSpec):
- spec = execnet.XSpec(spec)
- if not spec.chdir and not spec.popen:
- spec.chdir = defaultchdir
- self.specs.append(spec)
-
- def makegateways(self):
- assert not list(self.group)
- for spec in self.specs:
- gw = self.group.makegateway(spec)
- self.hook.pytest_gwmanage_newgateway(gateway=gw)
-
- def rsync(self, source, notify=None, verbose=False, ignores=None):
- """ perform rsync to all remote hosts.
- """
- rsync = HostRSync(source, verbose=verbose, ignores=ignores)
- seen = py.builtin.set()
- gateways = []
- for gateway in self.group:
- spec = gateway.spec
- if spec.popen and not spec.chdir:
- # XXX this assumes that sources are python-packages
- # and that adding the basedir does not hurt
- gateway.remote_exec("""
- import sys ; sys.path.insert(0, %r)
- """ % os.path.dirname(str(source))).waitclose()
- continue
- if spec not in seen:
- def finished():
- if notify:
- notify("rsyncrootready", spec, source)
- rsync.add_target_host(gateway, finished=finished)
- seen.add(spec)
- gateways.append(gateway)
- if seen:
- self.hook.pytest_gwmanage_rsyncstart(
- source=source,
- gateways=gateways,
- )
- rsync.send()
- self.hook.pytest_gwmanage_rsyncfinish(
- source=source,
- gateways=gateways,
- )
-
- def exit(self):
- self.group.terminate(self.EXIT_TIMEOUT)
-
-class HostRSync(execnet.RSync):
- """ RSyncer that filters out common files
- """
- def __init__(self, sourcedir, *args, **kwargs):
- self._synced = {}
- ignores= None
- if 'ignores' in kwargs:
- ignores = kwargs.pop('ignores')
- self._ignores = ignores or []
- super(HostRSync, self).__init__(sourcedir=sourcedir, **kwargs)
-
- def filter(self, path):
- path = py.path.local(path)
- if not path.ext in ('.pyc', '.pyo'):
- if not path.basename.endswith('~'):
- if path.check(dotfile=0):
- for x in self._ignores:
- if path == x:
- break
- else:
- return True
-
- def add_target_host(self, gateway, finished=None):
- remotepath = os.path.basename(self._sourcedir)
- super(HostRSync, self).add_target(gateway, remotepath,
- finishedcallback=finished,
- delete=True,)
-
- def _report_send_file(self, gateway, modified_rel_path):
- if self._verbose:
- path = os.path.basename(self._sourcedir) + "/" + modified_rel_path
- remotepath = gateway.spec.chdir
- py.builtin.print_('%s:%s <= %s' %
- (gateway.spec, remotepath, path))
--- a/testing/conftest.py
+++ b/testing/conftest.py
@@ -27,7 +27,7 @@ def getspecssh(config):
if spec.ssh:
if not py.path.local.sysfind("ssh"):
py.test.skip("command not found: ssh")
- return spec
+ return str(spec)
py.test.skip("need '--gx ssh=...'")
def getsocketspec(config):
--- a/setup.py
+++ b/setup.py
@@ -1,18 +1,10 @@
-"""
-py.test 'xdist' plugin for distributed testing and loop-on-failing modes.
-
-See http://pytest.org/plugin/xdist.html for documentation and, after
-installation of the ``pytest-xdist`` PyPI package, ``py.test -h``
-for the new options.
-"""
-
from setuptools import setup
setup(
name="pytest-xdist",
- version='1.5a8',
+ version='1.5a9',
description='py.test xdist plugin for distributed testing and loop-on-failing modes',
- long_description=__doc__,
+ long_description=open('README.txt').read(),
license='GPLv2 or later',
author='holger krekel and contributors',
author_email='py-dev at codespeak.net,holger at merlinux.eu',
--- a/README.txt
+++ b/README.txt
@@ -1,29 +1,170 @@
-py.test xdist plugin: distributed testing and loop failure
+xdist: pytest distributed testing plugin
===============================================================
-.. _`pytest-xdist respository`: http://bitbucket.org/hpk42/pytest-xdist
-.. _`pytest`: http://pytest.org
+The `pytest-xdist`_ plugin extends py.test with some unique
+test execution modes:
-The pytest-xdist plugin extends `py.test`_ to ad-hoc distribute test
-runs to multiple CPUs or remote machines. It requires setuptools
-or distribute which help to pull in the neccessary execnet and
-pytest-core dependencies.
+* Looponfail: run your tests repeatedly in a subprocess. After each run py.test
+ waits until a file in your project changes and then re-runs the previously
+ failing tests. This is repeated until all tests pass after which again
+ a full run is performed.
-Install the plugin locally with::
+* multiprocess Load-balancing: if you have multiple CPUs or hosts you can use
+ those for a combined test run. This allows to speed up
+ development or to use special resources of remote machines.
- python setup.py install
+* Multi-Platform coverage: you can specify different Python interpreters
+ or different platforms and run tests in parallel on all of them.
-or use the package in develope/in-place mode, particularly
-useful with a checkout of the `pytest-xdist repository`_::
+Before running tests remotely, ``py.test`` efficiently "rsyncs" your
+program source code to the remote place. All test results
+are reported back and displayed to your local terminal.
+You may specify different Python versions and interpreters.
+
+
+Installation
+-----------------------
+
+Install the plugin with::
+
+ easy_install pytest-xdist
+
+ # or
+
+ pip install pytest-xdist
+
+or use the package in develope/in-place mode with
+a checkout of the `pytest-xdist repository`_ ::
python setup.py develop
-or use one of::
+Usage examples
+---------------------
- easy_install pytest-xdist
+Speed up test runs by sending tests to multiple CPUs
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
- pip install pytest-xdist
+To send tests to multiple CPUs, type::
-for downloading and installing it in one go.
+ py.test -n NUM
-holger krekel <holger at merlinux eu>
+Especially for longer running tests or tests requiring
+a lot of IO this can lead to considerable speed ups.
+
+
+Running tests in a Python subprocess
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+To instantiate a python2.4 sub process and send tests to it, you may type::
+
+ py.test -d --tx popen//python=python2.4
+
+This will start a subprocess which is run with the "python2.4"
+Python interpreter, found in your system binary lookup path.
+
+If you prefix the --tx option value like this::
+
+ --tx 3*popen//python=python2.4
+
+then three subprocesses would be created and tests
+will be load-balanced across these three processes.
+
+
+Sending tests to remote SSH accounts
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+Suppose you have a package ``mypkg`` which contains some
+tests that you can successfully run locally. And you
+have a ssh-reachable machine ``myhost``. Then
+you can ad-hoc distribute your tests by typing::
+
+ py.test -d --tx ssh=myhostpopen --rsyncdir mypkg mypkg
+
+This will synchronize your ``mypkg`` package directory
+to an remote ssh account and then locally collect tests
+and send them to remote places for execution.
+
+You can specify multiple ``--rsyncdir`` directories
+to be sent to the remote side.
+
+**NOTE:** For py.test to collect and send tests correctly
+you not only need to make sure all code and tests
+directories are rsynced, but that any test (sub) directory
+also has an ``__init__.py`` file because internally
+py.test references tests as a fully qualified python
+module path. **You will otherwise get strange errors**
+during setup of the remote side.
+
+Sending tests to remote Socket Servers
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+Download the single-module `socketserver.py`_ Python program
+and run it like this::
+
+ python socketserver.py
+
+It will tell you that it starts listening on the default
+port. You can now on your home machine specify this
+new socket host with something like this::
+
+ py.test -d --tx socket=192.168.1.102:8888 --rsyncdir mypkg mypkg
+
+
+.. _`atonce`:
+
+Running tests on many platforms at once
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+The basic command to run tests on multiple platforms is::
+
+ py.test --dist=each --tx=spec1 --tx=spec2
+
+If you specify a windows host, an OSX host and a Linux
+environment this command will send each tests to all
+platforms - and report back failures from all platforms
+at once. The specifications strings use the `xspec syntax`_.
+
+.. _`xspec syntax`: http://codespeak.net/execnet/trunk/basics.html#xspec
+
+.. _`socketserver.py`: http://bitbucket.org/hpk42/execnet/raw/2af991418160/execnet/script/socketserver.py
+
+.. _`execnet`: http://codespeak.net/execnet
+
+Specifying test exec environments in an ini file
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+pytest (since version 2.0) supports ini-style cofiguration.
+You can for example make running with three subprocesses
+your default like this::
+
+ [pytest]
+ addopts = -n3
+
+You can also add default environments like this::
+
+ [pytest]
+ addopts = --tx ssh=myhost//python=python2.5 --tx ssh=myhost//python=python2.6
+
+and then just type::
+
+ py.test --dist=each
+
+to run tests in each of the environments.
+
+Specifying "rsync" dirs in an ini-file
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+In a ``tox.ini`` or ``setup.cfg`` file in your root project directory
+you may specify directories to include or to exclude in synchronisation::
+
+ [pytest]
+ rsyncdirs = . mypkg helperpkg
+ rsyncignore = .hg
+
+These directory specifications are relative to the directory
+where the configuration file was found.
+
+.. _`pytest-xdist`: http://pypi.python.org/pypi/pytest-xdist
+.. _`pytest-xdist repository`: http://bitbucket.org/hpk42/pytest-xdist
+.. _`pytest`: http://pytest.org
+
More information about the pytest-commit
mailing list