From holger at merlinux.eu Tue Nov 8 23:18:05 2011 From: holger at merlinux.eu (holger krekel) Date: Tue, 8 Nov 2011 22:18:05 +0000 Subject: [py-dev] Status of py.test's unit tests now ? In-Reply-To: <20111028210803.GF27936@merlinux.eu> References: <20111026070756.GB27936@merlinux.eu> <20111027111242.GC27936@merlinux.eu> <4EA94B44.50200@gmx.de> <20111028210803.GF27936@merlinux.eu> Message-ID: <20111108221805.GQ27936@merlinux.eu> Hi Pere, i think i fixed (most of) the issues behind the failures on OSX. could you do a "pip install -i http://pypi.testrun.org -U pytest" and rerun the tests and send along the full output if there remain issues? thanks, holger On Fri, Oct 28, 2011 at 21:08 +0000, holger krekel wrote: > On Fri, Oct 28, 2011 at 01:32 +0200, Pere Martir wrote: > > On Thu, Oct 27, 2011 at 2:15 PM, Ronny Pfannschmidt > > wrote: > > > On 10/27/2011 01:54 PM, Pere Martir wrote: > > >> By the way, where is the CI server please ? > > > http://hudson.testrun.org/job/pytest/ > > > > It seems to be a problem specific to Mac OS X. Since there is no Mac > > OS X slave, I suppose that the unit tests on this platform has not > > been paid much attention ? > > right, is the reason i asked for a build host in my last mail. > > > It's strange that if I only executed a subset of unit tests with -k, > > they don't fail. For example: > > > > python pytest.py -k TestGenerator > > > > But as you can see in the attachment of my previous post, they failed. > > It's also true for many other test suites, any clue ? > > > > on my OSX 10.8 all tests pass FWIW. > > > By the way, I fixed a problem. The failure of > > TestSession.test_parsearg is because on Mac OS X /var is actually a > > symbolic link of /private/var, and many other path under /. Patching > > TmpDir to return realpath fixes the problem. This doesn't fix many > > failures. "tox -e py26" still blocks at test_pdb forever, for example. > > > > I feel like that the other failures are due to the other bugs. Is it > > worthwhile spending time looking at the code ? Or it's probably the > > problem of my configuration/environment (not the source code) ? > > I don't know. could you send a new "py.test testing" log with the > x.realpath() patch applied? > > Is there a way to log into your OSX machine by chance? > > thanks, > holger > From holger at merlinux.eu Thu Nov 10 19:44:14 2011 From: holger at merlinux.eu (holger krekel) Date: Thu, 10 Nov 2011 18:44:14 +0000 Subject: [py-dev] tox-1.2: some bug fixes for the virtualenv-based generic test runner Message-ID: <20111110184414.GB7210@merlinux.eu> tox 1.2: the virtualenv-based test run automatizer =========================================================================== I am happy to announce tox 1.2, now using and depending on the latest virtualenv code and containing some bug fixes. TOX automates tedious test activities driven from a simple ``tox.ini`` file, including: * creation and management of different virtualenv environments with different Python interpreters * packaging and installing your package into each of them * running your test tool of choice, be it nose, py.test or unittest2 or other tools such as "sphinx" doc checks * testing dev packages against each other without needing to upload to PyPI It works well on virtually all Python interpreters that support virtualenv. Docs and examples are at: http://tox.testrun.org Installation: pip install -U tox code hosting and issue tracking on bitbucket: http://bitbucket.org/hpk42/tox best, Holger Krekel 1.2 compared to 1.1 --------------------- - remove the virtualenv.py that was distributed with tox and depend on virtualenv-1.6.4 (possible now since the latter fixes a few bugs that the inling tried to work around) - fix issue10: work around UnicodeDecodeError when inokving pip (thanks Marc Abramowitz) - fix a problem with parsing {posargs} in tox commands (spotted by goodwill) - fix the warning check for commands to be installed in testevironment (thanks Michael Foord for reporting) From holger at merlinux.eu Sat Nov 12 00:20:29 2011 From: holger at merlinux.eu (holger krekel) Date: Fri, 11 Nov 2011 23:20:29 +0000 Subject: [py-dev] upcoming 2.2 features / RFC Message-ID: <20111111232029.GC27920@merlinux.eu> Hi folks, i've just committed some new docs and changes to pytest to be released as "2.2.0" soon. I welcome any feedback including "makes sense", "nice but only if it would do XYZ ...", ... You can install the dev version via pip install -i http://pypi.testrun.org -U pytest and should get at least 2.2.0.dev7 (see py.test --version). It should be backward compatible almost completely except for pytest-xdist which you can upgrade via "pip install -i http://pypi.testrun.org -U pytest-xdist" There are two new features: --durations=N profiling: see how long your tests and fixtures take by looking at the "N" slowest tests. and improved marking for structuring and instrumenting your test suite: registering markers, a new "-m markexpr" option and "--markers" for showing registered markers, and "--strict" for treating unregistered markers as an error, see http://pytest.org/2.2.0.dev7/mark.html#mark for some more details. best, holger From holger at merlinux.eu Fri Nov 18 20:10:07 2011 From: holger at merlinux.eu (holger krekel) Date: Fri, 18 Nov 2011 19:10:07 +0000 Subject: [py-dev] pytest-2.2.0: test marking++, parametrization++ and duration profiling Message-ID: <20111118191007.GH27920@merlinux.eu> py.test 2.2.0: test marking++, parametrization++ and duration profiling =========================================================================== pytest-2.2.0 is a test-suite compatible release of the popular py.test testing tool. Plugins might need upgrades. It comes with these improvements: * easier and more powerful parametrization of tests: - new @pytest.mark.parametrize decorator to run tests with different arguments - new metafunc.parametrize() API for parametrizing arguments independently - see examples at http://pytest.org/latest/example/parametrize.html - NOTE that parametrize() related APIs are still a bit experimental and might change in future releases. * improved handling of test markers and refined marking mechanism: - "-m markexpr" option for selecting tests according to their mark - a new "markers" ini-variable for registering test markers for your project - the new "--strict" bails out with an error if using unregistered markers. - see examples at http://pytest.org/latest/example/markers.html * duration profiling: new "--duration=N" option showing the N slowest test execution or setup/teardown calls. This is most useful if you want to find out where your slowest test code is. * also 2.2.0 performs more eager calling of teardown/finalizers functions resulting in better and more accurate reporting when they fail Besides there is the usual set of bug fixes along with a cleanup of pytest's own test suite allowing it to run on a wider range of environments. For general information, see extensive docs with examples here: http://pytest.org/ If you want to install or upgrade pytest you might just type:: pip install -U pytest # or easy_install -U pytest Thanks to Ronny Pfannschmidt, David Burns, Jeff Donner, Daniel Nouri, Alfredo Doza and all who gave feedback or sent bug reports. best, holger krekel notes on incompatibility ------------------------------ While test suites should work unchanged you might need to upgrade plugins: * You need a new version of the pytest-xdist plugin (1.7) for distributing test runs. * Other plugins might need an upgrade if they implement the ``pytest_runtest_logreport`` hook which now is called unconditionally for the setup/teardown fixture phases of a test. You may choose to ignore setup/teardown failures by inserting "if rep.when != 'call': return" or something similar. Note that most code probably "just" works because the hook was already called for failing setup/teardown phases of a test so a plugin should have been ready to grok such reports already. Changes between 2.1.3 and 2.2.0 ---------------------------------------- - fix issue90: introduce eager tearing down of test items so that teardown function are called earlier. - add an all-powerful metafunc.parametrize function which allows to parametrize test function arguments in multiple steps and therefore from indepdenent plugins and palces. - add a @pytest.mark.parametrize helper which allows to easily call a test function with different argument values - Add examples to the "parametrize" example page, including a quick port of Test scenarios and the new parametrize function and decorator. - introduce registration for "pytest.mark.*" helpers via ini-files or through plugin hooks. Also introduce a "--strict" option which will treat unregistered markers as errors allowing to avoid typos and maintain a well described set of markers for your test suite. See exaples at http://pytest.org/latest/mark.html and its links. - issue50: introduce "-m marker" option to select tests based on markers (this is a stricter and more predictable version of '-k' in that "-m" only matches complete markers and has more obvious rules for and/or semantics. - new feature to help optimizing the speed of your tests: --durations=N option for displaying N slowest test calls and setup/teardown methods. - fix issue87: --pastebin now works with python3 - fix issue89: --pdb with unexpected exceptions in doctest work more sensibly - fix and cleanup pytest's own test suite to not leak FDs - fix issue83: link to generated funcarg list - fix issue74: pyarg module names are now checked against imp.find_module false positives - fix compatibility with twisted/trial-11.1.0 use cases