From kai at gronr.com Fri May 1 01:28:46 2015 From: kai at gronr.com (Kai Groner) Date: Thu, 30 Apr 2015 19:28:46 -0400 Subject: [pytest-dev] Working with other dependency injection systems Message-ID: Hi testing people! I'm trying to figure out how I can test a code base that uses an existing dependency injection system. I've run into two problems, and I have solutions for each of them but I think maybe there is a better way, so I'm looking for some advice. The first problem is how do I override how a test using this injector is called? I think I want to use the pytest_runtest_call hook, but it still tries to invoke the test without the injector. My current solution is to use the experimental hookwrapper mechanism to replace item.obj with a partially bound version, then restore it afterward. The second problem I'm having is the py.test fixture system is trying to resolve arguments that are provided by this other injector. How can I tell it that some of these don't need to be provided? My current solution is to blind py.test with a wrapper function with a *a, **kw signature. I use functools.wraps, so annotations are introspectable for the other injector, and delete the __wrapped__ attribute to prevent py.test from introspecting it. Is there a nicer, and perhaps less blunt, way to influence the funcarg fixture behaviors? I've tried a couple things with the pytest_collection_modifyitems hook, but haven't gotten anything that works yet. Other details to know about this injection system: - we want to build and teardown the injector with each test - we may want to configure the injector differently for some tests (possibly with fixture data from py.test) Thanks, Kai -------------- next part -------------- An HTML attachment was scrubbed... URL: From brianna.laugher at gmail.com Tue May 5 00:28:22 2015 From: brianna.laugher at gmail.com (Brianna Laugher) Date: Tue, 5 May 2015 00:28:22 +0200 Subject: [pytest-dev] adopt pytest month is over! Message-ID: Hi all, So adopt pytest month is officially over! I am working on surveys for the helpers and projects and will send them out this week. I hope everyone enjoyed it and maybe even learned something. One thing, the helpers for Nefertari (REST API framework built on Pyramid and ElasticSearch) are a bit stuck and could do with some more experienced eyes, specifically about how to approach testing a framework. Although the month is over Nefertari don't have much to show for it yet and it would be nice to see if we can help them get going with the basics. Would anyone be willing to help there? cheers, Brianna -- They've just been waiting in a mountain for the right moment: http://modernthings.org/ From holger at merlinux.eu Wed May 6 13:38:34 2015 From: holger at merlinux.eu (holger krekel) Date: Wed, 6 May 2015 11:38:34 +0000 Subject: [pytest-dev] adopt pytest month is over! In-Reply-To: References: Message-ID: <20150506113834.GP25957@merlinux.eu> Hi Brianna! thanks for this great initiative! I am also looking forward to the survey. Note sure i can help myself but are there some notes/summary of what is the problem with Nefertari currently? best, holger On Tue, May 05, 2015 at 00:28 +0200, Brianna Laugher wrote: > Hi all, > > So adopt pytest month is officially over! I am working on surveys for > the helpers and projects and will send them out this week. I hope > everyone enjoyed it and maybe even learned something. > > One thing, the helpers for Nefertari (REST API framework built on > Pyramid and ElasticSearch) are a bit stuck and could do with some more > experienced eyes, specifically about how to approach testing a > framework. Although the month is over Nefertari don't have much to > show for it yet and it would be nice to see if we can help them get > going with the basics. Would anyone be willing to help there? > > cheers, > Brianna > > -- > They've just been waiting in a mountain for the right moment: > http://modernthings.org/ > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > -- about me: http://holgerkrekel.net/about-me/ contracting: http://merlinux.eu From holger at merlinux.eu Thu May 7 13:21:07 2015 From: holger at merlinux.eu (holger krekel) Date: Thu, 7 May 2015 11:21:07 +0000 Subject: [pytest-dev] Working with other dependency injection systems In-Reply-To: References: Message-ID: <20150507112107.GA28354@merlinux.eu> Hi Kai, On Thu, Apr 30, 2015 at 19:28 -0400, Kai Groner wrote: > Hi testing people! > > I'm trying to figure out how I can test a code base that uses an existing > dependency injection system. I've run into two problems, and I have > solutions for each of them but I think maybe there is a better way, so I'm > looking for some advice. Could you provide a simple abstract example? Here and also in the following i don't really understand the background. holger > The first problem is how do I override how a test using this injector is > called? I think I want to use the pytest_runtest_call hook, but it still > tries to invoke the test without the injector. My current solution is to > use the experimental hookwrapper mechanism to replace item.obj with a > partially bound version, then restore it afterward. > > The second problem I'm having is the py.test fixture system is trying to > resolve arguments that are provided by this other injector. How can I tell > it that some of these don't need to be provided? My current solution is to > blind py.test with a wrapper function with a *a, **kw signature. I use > functools.wraps, so annotations are introspectable for the other injector, > and delete the __wrapped__ attribute to prevent py.test from introspecting > it. Is there a nicer, and perhaps less blunt, way to influence the funcarg > fixture behaviors? I've tried a couple things with the > pytest_collection_modifyitems hook, but haven't gotten anything that works > yet. > > Other details to know about this injection system: > - we want to build and teardown the injector with each test > - we may want to configure the injector differently for some tests > (possibly with fixture data from py.test) > > > > Thanks, > Kai > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev -- about me: http://holgerkrekel.net/about-me/ contracting: http://merlinux.eu From howaryoo at gmail.com Thu May 7 17:49:08 2015 From: howaryoo at gmail.com (=?UTF-8?Q?Ary=C3=A9_H?=) Date: Thu, 7 May 2015 18:49:08 +0300 Subject: [pytest-dev] adopt pytest month is over! Message-ID: Hi, Our question is on the right approach to testing such a framework ? 1) start with low level unit tests and strive to mock out the DB 2) continue with higher level integration tests with a DB and data fixtures More specifically (1) What is the best way to write unit tests that need model definitions (without database) and (2) integration tests that need database access in a framework that relies on Pyramid+SQLA but doesn't have any models defined in the code. Thanks for the follow up! Any advice would be greatly appreciated. > Message: 1 > Date: Wed, 6 May 2015 11:38:34 +0000 > From: holger krekel > To: Brianna Laugher > Cc: "pytest-dev at python.org" , Chris Hart > > Subject: Re: [pytest-dev] adopt pytest month is over! > Message-ID: <20150506113834.GP25957 at merlinux.eu> > Content-Type: text/plain; charset=us-ascii > > Hi Brianna! > > thanks for this great initiative! I am also looking forward to the > survey. > > Note sure i can help myself but are there some notes/summary of what is > the problem with Nefertari currently? > > best, > holger > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From florian.schulze at gmx.net Thu May 7 18:37:58 2015 From: florian.schulze at gmx.net (Florian Schulze) Date: Thu, 07 May 2015 18:37:58 +0200 Subject: [pytest-dev] adopt pytest month is over! In-Reply-To: References: Message-ID: Just from my experience testing more complex apps which have some framework enhancements. Don't overdo the mocking, it's far to easy to have tests which work fine with mocks, but the regular code with real objects fails. I tend to use mocking for edge cases only, where I want to test exception handling and the like. With pytest it's fairly easy to parametrize a fixture and provide a db connection with both in memory sqlite and something like postgres. For regular development you let the tests run quickly with in memory sqlite and before merging or just from time to time or via a CI system you test with the real db. Try to write your stuff in a way that you don't rely too much on the db side of models, then it's easier to replace them with mocks. I often use wrapper classes which add more functionality instead of adding that directly to the models. Composition instead of inheritance etc. IMO pyramid and pyramid-sqlalchemy have good examples on how to do testing with a framework. The latter has good support for savepoints, so you don't have to setup your db content all the time, but can just rollback after each test. AFAIK it support pytest fixtures, or at least provides helpers which make it easy to write fixtures. I haven't used pyramid-sqlalchemy myself yet, but in a project I'm involved in we use a pretty similar way to do the same thing. https://pyramid-sqlalchemy.readthedocs.org/en/latest/tests.html I haven't switched any of my app test suites to pytest yet. One I got working, but it's currently slower than zope.testrunner with layers, so it's postponed. The new pyramid-sqlalchemy test helpers should make that better though. I hope to be able to try again soon. I have converted quite a few other python packages of mine to pytest though. Regards, Florian Schulze On 7 May 2015, at 17:49, Ary? H wrote: > Hi, > > Our question is on the right approach to testing such a framework ? > > 1) start with low level unit tests and strive to mock out the DB > 2) continue with higher level integration tests with a DB and data > fixtures > > More specifically > > (1) What is the best way to write unit tests that need model > definitions > (without database) > and (2) integration tests that need database access in a framework > that > relies on Pyramid+SQLA but doesn't have any models defined in the > code. > > Thanks for the follow up! > Any advice would be greatly appreciated. > > > >> Message: 1 >> Date: Wed, 6 May 2015 11:38:34 +0000 >> From: holger krekel >> To: Brianna Laugher >> Cc: "pytest-dev at python.org" , Chris Hart >> >> Subject: Re: [pytest-dev] adopt pytest month is over! >> Message-ID: <20150506113834.GP25957 at merlinux.eu> >> Content-Type: text/plain; charset=us-ascii >> >> Hi Brianna! >> >> thanks for this great initiative! I am also looking forward to the >> survey. >> >> Note sure i can help myself but are there some notes/summary of what >> is >> the problem with Nefertari currently? >> >> best, >> holger >> >> > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev From schettino72 at gmail.com Fri May 8 10:29:33 2015 From: schettino72 at gmail.com (Eduardo Schettino) Date: Fri, 8 May 2015 16:29:33 +0800 Subject: [pytest-dev] [ANN] plugin pytest-incremental 0.4 released Message-ID: Hi, pytest-incremental is a plugin that aims to give faster feedback for test results. It analyses the source code (dependency) structure and keeps track of file modifications to: - re-order tests into a more relevant order than alphabetical order - de-select (skip) tests when changes since last execution dont affect the test This is the first release that features test re-ordering and adds python3 support. Though it is still in "alpha" stage (works for me), please let me know if you find any issues. pypi: https://pypi.python.org/pypi/pytest-incremental docs: http://pytest-incremental.readthedocs.org source: https://github.com/pytest-dev/pytest-incremental Regards, Eduardo -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Tue May 12 23:46:15 2015 From: holger at merlinux.eu (holger krekel) Date: Tue, 12 May 2015 21:46:15 +0000 Subject: [pytest-dev] tox-2.0.0: plugin support, platforms, env isolation Message-ID: <20150512214615.GN28354@merlinux.eu> tox-2.0: plugins, platform, env isolation ========================================== tox-2.0 was released to pypi, a major new release with *mostly* backward-compatible enhancements and fixes: - experimental support for plugins, see https://testrun.org/tox/dev/plugins.html which includes also a refined internal registration mechanism for new testenv ini options. You can now ask tox which testenv ini parameters exist with ``tox --help-ini``. - ENV isolation: only pass through very few environment variables from the tox invocation to the test environments. This may break test runs that previously worked with tox-1.9 -- you need to either use the ``setenv`` or ``passenv`` ini variables to set appropriate environment variables. - PLATFORM support: you can set ``platform=REGEX`` in your testenv sections which lets tox skip the environment if the REGEX does not match ``sys.platform``. - tox now stops execution of test commands if the first of them fails unless you set ``ignore_errors=True``. Thanks to Volodymyr Vitvitski, Daniel Hahler, Marc Abramowitz, Anthon van der Neuth and others for contributions. More documentation about tox in general: http://tox.testrun.org/ Installation: pip install -U tox code hosting and issue tracking on bitbucket: https://bitbucket.org/hpk42/tox What is tox? ---------------- tox standardizes and automates tedious test activities driven from a simple ``tox.ini`` file, including: * creation and management of different virtualenv environments with different Python interpreters * packaging and installing your package into each of them * running your test tool of choice, be it nose, py.test or unittest2 or other tools such as "sphinx" doc checks * testing dev packages against each other without needing to upload to PyPI best, Holger Krekel, merlinux GmbH -- about me: http://holgerkrekel.net/about-me/ contracting: http://merlinux.eu 2.0.0 ----------- - (new) introduce environment variable isolation: tox now only passes the PATH and PIP_INDEX_URL variable from the tox invocation environment to the test environment and on Windows also ``SYSTEMROOT``, ``PATHEXT``, ``TEMP`` and ``TMP`` whereas on unix additionally ``TMPDIR`` is passed. If you need to pass through further environment variables you can use the new ``passenv`` setting, a space-separated list of environment variable names. Each name can make use of fnmatch-style glob patterns. All environment variables which exist in the tox-invocation environment will be copied to the test environment. - a new ``--help-ini`` option shows all possible testenv settings and their defaults. - (new) introduce a way to specify on which platform a testenvironment is to execute: the new per-venv "platform" setting allows to specify a regular expression which is matched against sys.platform. If platform is set and doesn't match the platform spec in the test environment the test environment is ignored, no setup or tests are attempted. - (new) add per-venv "ignore_errors" setting, which defaults to False. If ``True``, a non-zero exit code from one command will be ignored and further commands will be executed (which was the default behavior in tox < 2.0). If ``False`` (the default), then a non-zero exit code from one command will abort execution of commands for that environment. - show and store in json the version dependency information for each venv - remove the long-deprecated "distribute" option as it has no effect these days. - fix issue233: avoid hanging with tox-setuptools integration example. Thanks simonb. - fix issue120: allow substitution for the commands section. Thanks Volodymyr Vitvitski. - fix issue235: fix AttributeError with --installpkg. Thanks Volodymyr Vitvitski. - tox has now somewhat pep8 clean code, thanks to Volodymyr Vitvitski. - fix issue240: allow to specify empty argument list without it being rewritten to ".". Thanks Daniel Hahler. - introduce experimental (not much documented yet) plugin system based on pytest's externalized "pluggy" system. See tox/hookspecs.py for the current hooks. - introduce parser.add_testenv_attribute() to register an ini-variable for testenv sections. Can be used from plugins through the tox_add_option hook. - rename internal files -- tox offers no external API except for the experimental plugin hooks, use tox internals at your own risk. - DEPRECATE distshare in documentation From holger at merlinux.eu Wed May 13 00:58:46 2015 From: holger at merlinux.eu (holger krekel) Date: Tue, 12 May 2015 22:58:46 +0000 Subject: [pytest-dev] [TIP] tox-2.0.0: plugin support, platforms, env isolation In-Reply-To: References: <20150512214615.GN28354@merlinux.eu> Message-ID: <20150512225846.GP28354@merlinux.eu> Hi Alex, On Tue, May 12, 2015 at 18:51 -0400, Alex Gaynor wrote: > Hi Holger, > > First: congrats on shipping 2.0, I'm very excited to explore some of the > new features. > > Second: > > Did tox drop Python 2.6 support? It started failing on our CI, so far the > error I see is that tox imports argparse, but I guess it's setup.py does > not depend on it. Can this be fixed? Hum, that's probably a problem of the wheel format. Can you try installing with "--no-use-wheel" and see if that helps? holger > Thanks! > Alex > > On Tue, May 12, 2015 at 5:46 PM, holger krekel wrote: > > > tox-2.0: plugins, platform, env isolation > > ========================================== > > > > tox-2.0 was released to pypi, a major new release with *mostly* > > backward-compatible enhancements and fixes: > > > > - experimental support for plugins, see > > https://testrun.org/tox/dev/plugins.html > > which includes also a refined internal registration mechanism for new > > testenv > > ini options. You can now ask tox which testenv ini parameters exist > > with ``tox --help-ini``. > > > > - ENV isolation: only pass through very few environment variables from the > > tox invocation to the test environments. This may break test runs that > > previously worked with tox-1.9 -- you need to either use the > > ``setenv`` or ``passenv`` ini variables to set appropriate environment > > variables. > > > > - PLATFORM support: you can set ``platform=REGEX`` in your testenv sections > > which lets tox skip the environment if the REGEX does not match > > ``sys.platform``. > > > > - tox now stops execution of test commands if the first of them fails > > unless > > you set ``ignore_errors=True``. > > > > Thanks to Volodymyr Vitvitski, Daniel Hahler, Marc Abramowitz, Anthon van > > der Neuth and others for contributions. > > > > More documentation about tox in general: > > > > http://tox.testrun.org/ > > > > Installation: > > > > pip install -U tox > > > > code hosting and issue tracking on bitbucket: > > > > https://bitbucket.org/hpk42/tox > > > > What is tox? > > ---------------- > > > > tox standardizes and automates tedious test activities driven from a > > simple ``tox.ini`` file, including: > > > > * creation and management of different virtualenv environments > > with different Python interpreters > > * packaging and installing your package into each of them > > * running your test tool of choice, be it nose, py.test or unittest2 or > > other tools such as "sphinx" doc checks > > * testing dev packages against each other without needing to upload to PyPI > > > > best, > > Holger Krekel, merlinux GmbH > > > > -- > > about me: http://holgerkrekel.net/about-me/ > > contracting: http://merlinux.eu > > > > 2.0.0 > > ----------- > > > > - (new) introduce environment variable isolation: > > tox now only passes the PATH and PIP_INDEX_URL variable from the tox > > invocation environment to the test environment and on Windows > > also ``SYSTEMROOT``, ``PATHEXT``, ``TEMP`` and ``TMP`` whereas > > on unix additionally ``TMPDIR`` is passed. If you need to pass > > through further environment variables you can use the new ``passenv`` > > setting, > > a space-separated list of environment variable names. Each name > > can make use of fnmatch-style glob patterns. All environment > > variables which exist in the tox-invocation environment will be copied > > to the test environment. > > > > - a new ``--help-ini`` option shows all possible testenv settings and > > their defaults. > > > > - (new) introduce a way to specify on which platform a testenvironment is > > to > > execute: the new per-venv "platform" setting allows to specify > > a regular expression which is matched against sys.platform. > > If platform is set and doesn't match the platform spec in the test > > environment the test environment is ignored, no setup or tests are > > attempted. > > > > - (new) add per-venv "ignore_errors" setting, which defaults to False. > > If ``True``, a non-zero exit code from one command will be ignored and > > further commands will be executed (which was the default behavior in > > tox < > > 2.0). If ``False`` (the default), then a non-zero exit code from one > > command > > will abort execution of commands for that environment. > > > > - show and store in json the version dependency information for each venv > > > > - remove the long-deprecated "distribute" option as it has no effect these > > days. > > > > - fix issue233: avoid hanging with tox-setuptools integration example. > > Thanks simonb. > > > > - fix issue120: allow substitution for the commands section. Thanks > > Volodymyr Vitvitski. > > > > - fix issue235: fix AttributeError with --installpkg. Thanks > > Volodymyr Vitvitski. > > > > - tox has now somewhat pep8 clean code, thanks to Volodymyr Vitvitski. > > > > - fix issue240: allow to specify empty argument list without it being > > rewritten to ".". Thanks Daniel Hahler. > > > > - introduce experimental (not much documented yet) plugin system > > based on pytest's externalized "pluggy" system. > > See tox/hookspecs.py for the current hooks. > > > > - introduce parser.add_testenv_attribute() to register an ini-variable > > for testenv sections. Can be used from plugins through the > > tox_add_option hook. > > > > - rename internal files -- tox offers no external API except for the > > experimental plugin hooks, use tox internals at your own risk. > > > > - DEPRECATE distshare in documentation > > > > > > _______________________________________________ > > testing-in-python mailing list > > testing-in-python at lists.idyll.org > > http://lists.idyll.org/listinfo/testing-in-python > > > > > > -- > "I disapprove of what you say, but I will defend to the death your right to > say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 -- about me: http://holgerkrekel.net/about-me/ contracting: http://merlinux.eu From ken.petti at cloudlock.com Wed May 13 21:42:13 2015 From: ken.petti at cloudlock.com (Ken Petti) Date: Wed, 13 May 2015 15:42:13 -0400 Subject: [pytest-dev] Fixture override not working Message-ID: Hello all, Last week I spoke to Ronny on the IRC channel about a fixture issue, and he recommended I send out an email to the larger group. Here is a gist of the issuehttps://gist.github.com/citizenken/861055b8778c9cce343a I have a fixture on the module level that I want to override on a per-test basis. This module fixture handles app log in and is both autoused and function scoped. I override it with a tuple value, to use non-default credentials, and then use the new fixture in an alternate login fixture. What I expect to happen is that the module fixture it overridden with my static values, the alternate login fixture then uses those static values and logs in, then control goes to my tests, which executes as expected. What is happening is that the module fixture is never overridden, so the app logs in with the wrong credentials and the test fails. Ronny suggested modifying my gist and using hidden fixtures like so: https://gist.github.com/RonnyPfannschmidt/3c17b07aa56d6231d38d This did not work. Instead, the hidden fixture was executed with the static values, then, without closing the browser, the original login_by_account fixture is called. This is an excerpt from my chat with Ronny: [13:25] citken: i think you missunderstood me, the idea is to separate th data fixture and the autouse fixture, since it seems all autouse instaces are made n all cases [13:27] ronny: can you show me an example/comment on the gist what i should do? [13:28] citken: https://gist.github.com/RonnyPfannschmidt/3c17b07aa56d6231d38d [13:30] ok let me try that now [13:30] thanks, that makes more sense [13:31] citken: could you please repost an issue about autouse fixture overriding issues [13:34] yeah that isn't working either [13:34] my fixture is overriding for all test cases [13:35] it should only override for the testcase that is marked parameterize [13:35] making the hidden fixture, as in your gist, does not work correctly [13:43] then im missing something [13:44] citken: what is the problem with the hidden fixture/ [14:01] ronny: so i don't think your gist is handling the override correctly [14:02] citken: im probably missing something, what exactly is happening how? [14:04] its weird... it looks like _ensure_login is hit first (expected, with autouse=True), and that has the value of my override. My app logs in with the override credentials. Then, without closing the browser or anything, the original login_by_account fixture is called, the one with the scope of "function" [14:04] is the stack trace like this?: [14:05] override -> autouse fixture -> function scope fixture -> test? [14:07] citken: not sure, i dont know the exact details, holger and bubenkoff worked with fixture oerrides in-depth, and currently im at the end of what i can help with If anyone could give me a hand that would be great! Thanks! -- *Ken Petti* Software Development Engineer in Test E-Mail: ken.petti at cloudlock.com *CloudLock provides compliance and security solutions for enterprises using public cloud applications like Salesforce and Google Apps. The largest organizations in the world trust CloudLock to secure their data*. -------------- next part -------------- An HTML attachment was scrubbed... URL: From flub at devork.be Tue May 19 10:40:55 2015 From: flub at devork.be (Floris Bruynooghe) Date: Tue, 19 May 2015 09:40:55 +0100 Subject: [pytest-dev] pytest-2.7.1: bugfix release, wheel support, yield fixtures Message-ID: <20150519084055.GA24198@laurie.devork.be> pytest-2.7.1: bugfix release, wheel support and yield fixtures ============================================================== pytest is a mature Python testing tool with more than a 1100 tests against itself, passing on many different interpreters and platforms. This release is supposed to be drop-in compatible to 2.7.0. The most user-visible changes are that wheels are now available on pypi and yield fixtures are no longer marked as experimental, they are here to stay. See below for the changes and see docs at: http://pytest.org As usual, you can upgrade from pypi via:: pip install -U pytest Thanks to all who contributed to this release, among them: Bruno Oliveira Holger Krekel Ionel Maries Cristian Floris Bruynooghe Happy testing, The py.test Development Team 2.7.1 (compared to 2.7.0) ------------------------- - fix issue731: do not get confused by the braces which may be present and unbalanced in an object's repr while collapsing False explanations. Thanks Carl Meyer for the report and test case. - fix issue553: properly handling inspect.getsourcelines failures in FixtureLookupError which would lead to to an internal error, obfuscating the original problem. Thanks talljosh for initial diagnose/patch and Bruno Oliveira for final patch. - fix issue660: properly report scope-mismatch-access errors independently from ordering of fixture arguments. Also avoid the pytest internal traceback which does not provide information to the user. Thanks Holger Krekel. - streamlined and documented release process. Also all versions (in setup.py and documentation generation) are now read from _pytest/__init__.py. Thanks Holger Krekel. - fixed docs to remove the notion that yield-fixtures are experimental. They are here to stay :) Thanks Bruno Oliveira. - Support building wheels by using environment markers for the requirements. Thanks Ionel Maries Cristian. - fixed regression to 2.6.4 which surfaced e.g. in lost stdout capture printing when tests raised SystemExit. Thanks Holger Krekel. - reintroduced _pytest fixture of the pytester plugin which is used at least by pytest-xdist. From raphael at hackebrot.de Thu May 21 09:17:31 2015 From: raphael at hackebrot.de (raphael at hackebrot.de) Date: Thu, 21 May 2015 09:17:31 +0200 Subject: [pytest-dev] tox-2.0.0: plugin support, platforms, env isolation In-Reply-To: <20150512214615.GN28354@merlinux.eu> References: <20150512214615.GN28354@merlinux.eu> Message-ID: Hi testing folks, I have been running into a slightly confusing problem on my Cookiecutter dev env with the latest tox version and click. The changes regarding ENV isolation happen to break our tox suite on Python3 envs because Python is making a guess on the preferred encoding which happens to be 'ascii' which in turn causes a 'RuntimeError' in click. Please see the according PR and the gist included. https://github.com/audreyr/cookiecutter/pull/453 Given that this is a deliberate change in tox 2.0, I was thinking whether we should spread the word (possibly reddit, twitter?!) so other projects affected by this change are aware and update their tox config accordingly. Please let me know your thoughts and I'll be happy to contribute. Best Raphael Am 12.05.2015 23:46, schrieb holger krekel: > tox-2.0: plugins, platform, env isolation > ========================================== > > tox-2.0 was released to pypi, a major new release with *mostly* > backward-compatible enhancements and fixes: > > - experimental support for plugins, see > https://testrun.org/tox/dev/plugins.html > which includes also a refined internal registration mechanism for new > testenv > ini options. You can now ask tox which testenv ini parameters exist > with ``tox --help-ini``. > > - ENV isolation: only pass through very few environment variables from > the > tox invocation to the test environments. This may break test runs > that > previously worked with tox-1.9 -- you need to either use the > ``setenv`` or ``passenv`` ini variables to set appropriate > environment > variables. > > - PLATFORM support: you can set ``platform=REGEX`` in your testenv > sections > which lets tox skip the environment if the REGEX does not match > ``sys.platform``. > > - tox now stops execution of test commands if the first of them fails > unless > you set ``ignore_errors=True``. > > Thanks to Volodymyr Vitvitski, Daniel Hahler, Marc Abramowitz, Anthon > van > der Neuth and others for contributions. > > More documentation about tox in general: > > http://tox.testrun.org/ > > Installation: > > pip install -U tox > > code hosting and issue tracking on bitbucket: > > https://bitbucket.org/hpk42/tox > > What is tox? > ---------------- > > tox standardizes and automates tedious test activities driven from a > simple ``tox.ini`` file, including: > > * creation and management of different virtualenv environments > with different Python interpreters > * packaging and installing your package into each of them > * running your test tool of choice, be it nose, py.test or unittest2 > or other tools such as "sphinx" doc checks > * testing dev packages against each other without needing to upload to > PyPI > > best, > Holger Krekel, merlinux GmbH > > -- > about me: http://holgerkrekel.net/about-me/ > contracting: http://merlinux.eu > > 2.0.0 > ----------- > > - (new) introduce environment variable isolation: > tox now only passes the PATH and PIP_INDEX_URL variable from the tox > invocation environment to the test environment and on Windows > also ``SYSTEMROOT``, ``PATHEXT``, ``TEMP`` and ``TMP`` whereas > on unix additionally ``TMPDIR`` is passed. If you need to pass > through further environment variables you can use the new ``passenv`` > setting, > a space-separated list of environment variable names. Each name > can make use of fnmatch-style glob patterns. All environment > variables which exist in the tox-invocation environment will be > copied > to the test environment. > > - a new ``--help-ini`` option shows all possible testenv settings and > their defaults. > > - (new) introduce a way to specify on which platform a testenvironment > is to > execute: the new per-venv "platform" setting allows to specify > a regular expression which is matched against sys.platform. > If platform is set and doesn't match the platform spec in the test > environment the test environment is ignored, no setup or tests are > attempted. > > - (new) add per-venv "ignore_errors" setting, which defaults to False. > If ``True``, a non-zero exit code from one command will be ignored > and > further commands will be executed (which was the default behavior in > tox < > 2.0). If ``False`` (the default), then a non-zero exit code from > one command > will abort execution of commands for that environment. > > - show and store in json the version dependency information for each > venv > > - remove the long-deprecated "distribute" option as it has no effect > these days. > > - fix issue233: avoid hanging with tox-setuptools integration example. > Thanks simonb. > > - fix issue120: allow substitution for the commands section. Thanks > Volodymyr Vitvitski. > > - fix issue235: fix AttributeError with --installpkg. Thanks > Volodymyr Vitvitski. > > - tox has now somewhat pep8 clean code, thanks to Volodymyr Vitvitski. > > - fix issue240: allow to specify empty argument list without it being > rewritten to ".". Thanks Daniel Hahler. > > - introduce experimental (not much documented yet) plugin system > based on pytest's externalized "pluggy" system. > See tox/hookspecs.py for the current hooks. > > - introduce parser.add_testenv_attribute() to register an ini-variable > for testenv sections. Can be used from plugins through the > tox_add_option hook. > > - rename internal files -- tox offers no external API except for the > experimental plugin hooks, use tox internals at your own risk. > > - DEPRECATE distshare in documentation > > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev From mail at florian-schulze.net Thu May 21 10:16:03 2015 From: mail at florian-schulze.net (Florian Schulze) Date: Thu, 21 May 2015 10:16:03 +0200 Subject: [pytest-dev] tox-2.0.0: plugin support, platforms, env isolation In-Reply-To: References: <20150512214615.GN28354@merlinux.eu> Message-ID: <251C5B25-A53B-4667-9A34-15348C7FC7B6@florian-schulze.net> On 21 May 2015, at 9:17, raphael at hackebrot.de wrote: > Hi testing folks, > > I have been running into a slightly confusing problem on my > Cookiecutter dev env with the latest tox version and click. > > The changes regarding ENV isolation happen to break our tox suite on > Python3 envs because Python is making a guess on the preferred > encoding which happens to be 'ascii' which in turn causes a > 'RuntimeError' in click. > > Please see the according PR and the gist included. > https://github.com/audreyr/cookiecutter/pull/453 > > Given that this is a deliberate change in tox 2.0, I was thinking > whether we should spread the word (possibly reddit, twitter?!) so > other projects affected by this change are aware and update their tox > config accordingly. > > Please let me know your thoughts and I'll be happy to contribute. I just hit the same issue, but for me it happens during installation because of unicode in the long_description. With passenv = LC_ALL, LANG it works. Question is, should these envvars be added by default or does everyone have to update their tox.ini? Regards, Florian Schulze From me at the-compiler.org Thu May 21 10:52:28 2015 From: me at the-compiler.org (Florian Bruhin) Date: Thu, 21 May 2015 10:52:28 +0200 Subject: [pytest-dev] tox-2.0.0: plugin support, platforms, env isolation In-Reply-To: <251C5B25-A53B-4667-9A34-15348C7FC7B6@florian-schulze.net> References: <20150512214615.GN28354@merlinux.eu> <251C5B25-A53B-4667-9A34-15348C7FC7B6@florian-schulze.net> Message-ID: <20150521085228.GR448@tonks> * Florian Schulze [2015-05-21 10:16:03 +0200]: > On 21 May 2015, at 9:17, raphael at hackebrot.de wrote: > > >Hi testing folks, > > > >I have been running into a slightly confusing problem on my Cookiecutter > >dev env with the latest tox version and click. > > > >The changes regarding ENV isolation happen to break our tox suite on > >Python3 envs because Python is making a guess on the preferred encoding > >which happens to be 'ascii' which in turn causes a 'RuntimeError' in > >click. > > > >Please see the according PR and the gist included. > >https://github.com/audreyr/cookiecutter/pull/453 > > > >Given that this is a deliberate change in tox 2.0, I was thinking whether > >we should spread the word (possibly reddit, twitter?!) so other projects > >affected by this change are aware and update their tox config accordingly. > > > >Please let me know your thoughts and I'll be happy to contribute. > > I just hit the same issue, but for me it happens during installation because > of unicode in the long_description. With passenv = LC_ALL, LANG it works. > Question is, should these envvars be added by default or does everyone have > to update their tox.ini? The same thing broke my pep257 checks (docstring convention checker) as well as the pyflakes checks for pytest itself: https://bitbucket.org/pytest-dev/pytest/issue/747/flakes-test-fail I'm still not sure if it's a good idea to include LC_ALL/LANG by default though, even with stuff breaking. As mentioned by someone else in some mailinglist thread earlier, it's nice to know when tests depend on those variables. After all, there still could be people running the tests with a broken system locale and no UTF-8 support (I've gotten bugreports because of that!). In the usual cases, I think the code should be fixed instead: - pep257 and flake8 should use tokenize.detect_encoding[1] / tokenize.open (or equivalent code for < 3.2) to honour PEP 263 encoding markers[2]. (I got bitten by this when trying to run pep257 on Python files containing UTF-8 Unicode chars on Windows) - Code reading/writing files only used by itself should always specify an explicit encoding, probably encoding='utf-8' [3] (I got bitten by this by writing a cookie file with Unicode in cookies without explicit encoding - it worked on Linux but crashed on Windows because it used latin1) - Code writing files for other applications should either know the encoding the other part needs, or fallback to an explicit default (i.e. utf-8). - Code reading files from other applications should either know the encoding the other part writes, or find out by some kind of meta-data, or do a more elaborated guess (e.g. using chardet[4]). [1] https://docs.python.org/3.4/library/tokenize.html#tokenize.detect_encoding [2] https://www.python.org/dev/peps/pep-0263/ [3] http://utf8everywhere.org/ [4] https://pypi.python.org/pypi/chardet Does this sound reasonable? I know it's the much more painful way compared to basically ignoring those issues (by passing the locale through, which is most likely UTF8-aware on Linux). But I think it's better to fix the real issues wherever possible. Florian -- http://www.the-compiler.org | me at the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From mail at florian-schulze.net Thu May 21 11:33:58 2015 From: mail at florian-schulze.net (Florian Schulze) Date: Thu, 21 May 2015 11:33:58 +0200 Subject: [pytest-dev] tox-2.0.0: plugin support, platforms, env isolation In-Reply-To: <20150521085228.GR448@tonks> References: <20150512214615.GN28354@merlinux.eu> <251C5B25-A53B-4667-9A34-15348C7FC7B6@florian-schulze.net> <20150521085228.GR448@tonks> Message-ID: <9A9B3871-D0F1-4AB6-B22D-21070054C1FB@florian-schulze.net> Sounds reasonable, thanks for the list! Maybe we should put that somewhere findable? The tox docs, or at least link from there? Regards, Florian Schulze On 21 May 2015, at 10:52, Florian Bruhin wrote: > * Florian Schulze [2015-05-21 10:16:03 > +0200]: >> On 21 May 2015, at 9:17, raphael at hackebrot.de wrote: >> >>> Hi testing folks, >>> >>> I have been running into a slightly confusing problem on my >>> Cookiecutter >>> dev env with the latest tox version and click. >>> >>> The changes regarding ENV isolation happen to break our tox suite on >>> Python3 envs because Python is making a guess on the preferred >>> encoding >>> which happens to be 'ascii' which in turn causes a 'RuntimeError' in >>> click. >>> >>> Please see the according PR and the gist included. >>> https://github.com/audreyr/cookiecutter/pull/453 >>> >>> Given that this is a deliberate change in tox 2.0, I was thinking >>> whether >>> we should spread the word (possibly reddit, twitter?!) so other >>> projects >>> affected by this change are aware and update their tox config >>> accordingly. >>> >>> Please let me know your thoughts and I'll be happy to contribute. >> >> I just hit the same issue, but for me it happens during installation >> because >> of unicode in the long_description. With passenv = LC_ALL, LANG it >> works. >> Question is, should these envvars be added by default or does >> everyone have >> to update their tox.ini? > > The same thing broke my pep257 checks (docstring convention checker) > as well as the pyflakes checks for pytest itself: > > https://bitbucket.org/pytest-dev/pytest/issue/747/flakes-test-fail > > I'm still not sure if it's a good idea to include LC_ALL/LANG by > default though, even with stuff breaking. As mentioned by someone else > in some mailinglist thread earlier, it's nice to know when tests > depend on those variables. > > After all, there still could be people running the tests with a broken > system locale and no UTF-8 support (I've gotten bugreports because of > that!). > > In the usual cases, I think the code should be fixed instead: > > - pep257 and flake8 should use tokenize.detect_encoding[1] / > tokenize.open (or equivalent code for < 3.2) to honour PEP 263 > encoding markers[2]. > > (I got bitten by this when trying to run pep257 on Python files > containing UTF-8 Unicode chars on Windows) > > - Code reading/writing files only used by itself should always specify > an explicit encoding, probably encoding='utf-8' [3] > > (I got bitten by this by writing a cookie file with Unicode in > cookies without explicit encoding - it worked on Linux but crashed > on Windows because it used latin1) > > - Code writing files for other applications should either know the > encoding the other part needs, or fallback to an explicit default > (i.e. utf-8). > > - Code reading files from other applications should either know the > encoding the other part writes, or find out by some kind of > meta-data, or do a more elaborated guess (e.g. using chardet[4]). > > [1] > https://docs.python.org/3.4/library/tokenize.html#tokenize.detect_encoding > [2] https://www.python.org/dev/peps/pep-0263/ > [3] http://utf8everywhere.org/ > [4] https://pypi.python.org/pypi/chardet > > Does this sound reasonable? I know it's the much more painful way > compared to basically ignoring those issues (by passing the locale > through, which is most likely UTF8-aware on Linux). But I think it's > better to fix the real issues wherever possible. > > Florian > > -- > http://www.the-compiler.org | me at the-compiler.org (Mail/XMPP) > GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc > I love long mails! | http://email.is-not-s.ms/ > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev From me at the-compiler.org Thu May 21 13:57:07 2015 From: me at the-compiler.org (Florian Bruhin) Date: Thu, 21 May 2015 13:57:07 +0200 Subject: [pytest-dev] tox-2.0.0: plugin support, platforms, env isolation In-Reply-To: <9A9B3871-D0F1-4AB6-B22D-21070054C1FB@florian-schulze.net> <20150521085228.GR448@tonks> Message-ID: <20150521115707.GV448@tonks> * Florian Bruhin [2015-05-21 10:52:28 +0200]: > - pep257 and flake8 should use tokenize.detect_encoding[1] / > tokenize.open (or equivalent code for < 3.2) to honour PEP 263 > encoding markers[2]. > > (I got bitten by this when trying to run pep257 on Python files > containing UTF-8 Unicode chars on Windows) I've now fixed this for pep257 (at least for python3 - it seems to work fine on python2): https://github.com/GreenSteam/pep257/pull/118 I still have to look into pyflakes as I've only seen that failing in the pytest tests, but not with a minimal example. * Florian Schulze [2015-05-21 11:33:58 +0200]: > Sounds reasonable, thanks for the list! Maybe we should put that somewhere > findable? The tox docs, or at least link from there? If others agree - sure! Though I'd prefer someone other than me doing that ;) Florian (the Bruhin one :P) -- http://www.the-compiler.org | me at the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From me at the-compiler.org Fri May 22 08:03:01 2015 From: me at the-compiler.org (Florian Bruhin) Date: Fri, 22 May 2015 08:03:01 +0200 Subject: [pytest-dev] tox-2.0.0: plugin support, platforms, env isolation In-Reply-To: <20150521115707.GV448@tonks> References: <9A9B3871-D0F1-4AB6-B22D-21070054C1FB@florian-schulze.net> <20150521085228.GR448@tonks> <20150521115707.GV448@tonks> Message-ID: <20150522060301.GA448@tonks> * Florian Bruhin [2015-05-21 13:57:07 +0200]: > * Florian Bruhin [2015-05-21 10:52:28 +0200]: > > - pep257 and flake8 should use tokenize.detect_encoding[1] / > > tokenize.open (or equivalent code for < 3.2) to honour PEP 263 > > encoding markers[2]. > > > > (I got bitten by this when trying to run pep257 on Python files > > containing UTF-8 Unicode chars on Windows) For flake8 that seems to be an issue with pytest-flake8, not the upstream flake8. I opened an issue here, as I'm not sure what to do with python2: https://github.com/fschulze/pytest-flakes/issues/6 Florian -- http://www.the-compiler.org | me at the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From gherman at darwin.in-berlin.de Sun May 24 22:36:07 2015 From: gherman at darwin.in-berlin.de (Dinu Gherman) Date: Sun, 24 May 2015 22:36:07 +0200 Subject: [pytest-dev] Test support for Python micro versions? Message-ID: <20150524223607.Horde.76Icfy3dpqD0Ad0eW7_Bbg7@webmail.in-berlin.de> Hello! I'm looking for information about how to run py.test (or rather tox) on micro versions of Python releases (the micro version being part C in the full versioning scheme A.B.C, i.e. major.minor.micro, see also https://docs.python.org/devguide/devcycle.html). I've put a well-known search engine at work and have searched on pytest.org, but haven't been really successful. Is this a non-feature or just considered a non-issue? The latter would be surprising to me as Python micro releases have recently shown what I'd call significant changes, e.g. those concerning SSL between 2.7.8 and 2.7.9. So AFAICT it would be helpful to be able to run something like this: tox -e py278 py279 py2710 Any clues are highly appreciated, thanks! Cheers, Dinu