From holger at merlinux.eu Sun May 2 15:12:13 2010 From: holger at merlinux.eu (holger krekel) Date: Sun, 2 May 2010 15:12:13 +0200 Subject: [py-dev] execnet-1.0.6 released (jython/win32 and race condition fixes) Message-ID: <20100502131213.GD22675@trillke.net> execnet-1.0.6 is a backward compatible release fixing jython/win32 and general race condition issues. execnet is a small, stable and well documented pure-python library for automatically deploying and interacting with clusters of Python interpreters. It seamlessly instantiates remote interpreters through the 'ssh' command line tool or socket connections. It supports interactions between Python 2.4 through to 3.1, Jython-2.5.1 and pypy-c, therefore enabling generic Python-to-Python bridging. More info here: http://codespeak.net/execnet/ cheers, holger From holger at merlinux.eu Tue May 4 17:54:15 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 4 May 2010 17:54:15 +0200 Subject: [py-dev] extending "-p" / plugin pre-loading? Message-ID: <20100504155415.GC17693@trillke.net> Hi all, i am preparing a py/test-1.3.0 release and stumbled across the need and idea to extend the '-p' option - comprising these usages: py.test -p NAME # import plugin with the given name py.test -p path/to/conftest.py # pre-load this conftest.py file py.test -p path/to/NAME.py # load given file as a plugin The first line works with py-1.2.1 already. The other two would be new and useful if you have some special plugin/test running modes you only want to enable explicitely. The specified conftest/plugin code gets loaded before command line parsing so it can add new cmdline options already. However it turns out I don't currently have the need myself for this extension anymore and wonder if anybody is interested in this. If not i just keep my local patch in some attic. best, holger From holger at merlinux.eu Wed May 5 21:45:33 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 5 May 2010 21:45:33 +0200 Subject: [py-dev] py.test-1.3.0: new options, per-plugin hooks, better reporting Message-ID: <20100505194533.GP17693@trillke.net> py.test/pylib 1.3.0: new options, per-plugin hooks, fixes ... =========================================================================== The py.test/pylib 1.3.0 release introces new options, bug fixes and improved compatibility with Python3 and Jython-2.5.1 on Windows. If you already use py-1.2 chances are you can just use py-1.3.0. See the below CHANGELOG for more details and http://pylib.org/install.html for installation instructions. py.test is an advanced automated testing tool working with Python2, Python3, Jython and PyPy versions on all major operating systems. It offers a no-boilerplate testing approach and has inspired other testing tools and enhancements in the standard Python library for more than five years. It has a simple and extensive plugin architecture, configurable reporting and provides unique ways to make it fit to your testing process and needs. See http://pytest.org for more info. cheers and have fun, holger krekel Changes between 1.2.1 and 1.3.0 ================================================== - deprecate --report option in favour of a new shorter and easier to remember -r option: it takes a string argument consisting of any combination of 'xfsX' characters. They relate to the single chars you see during the dotted progress printing and will print an extra line per test at the end of the test run. This extra line indicates the exact position or test ID that you directly paste to the py.test cmdline in order to re-run a particular test. - allow external plugins to register new hooks via the new pytest_addhooks(pluginmanager) hook. The new release of the pytest-xdist plugin for distributed and looponfailing testing requires this feature. - add a new pytest_ignore_collect(path, config) hook to allow projects and plugins to define exclusion behaviour for their directory structure - for example you may define in a conftest.py this method:: def pytest_ignore_collect(path): return path.check(link=1) to prevent even collection of any tests in symlinked dirs. - new pytest_pycollect_makemodule(path, parent) hook for allowing customization of the Module collection object for a matching test module. - extend and refine xfail mechanism: ``@py.test.mark.xfail(run=False)`` do not run the decorated test ``@py.test.mark.xfail(reason="...")`` prints the reason string in xfail summaries specifiying ``--runxfail`` on command line virtually ignores xfail markers - expose (previously internal) commonly useful methods: py.io.get_terminal_with() -> return terminal width py.io.ansi_print(...) -> print colored/bold text on linux/win32 py.io.saferepr(obj) -> return limited representation string - expose test outcome related exceptions as py.test.skip.Exception, py.test.raises.Exception etc., useful mostly for plugins doing special outcome interpretation/tweaking - (issue85) fix junitxml plugin to handle tests with non-ascii output - fix/refine python3 compatibility (thanks Benjamin Peterson) - fixes for making the jython/win32 combination work, note however: jython2.5.1/win32 does not provide a command line launcher, see http://bugs.jython.org/issue1491 . See pylib install documentation for how to work around. - fixes for handling of unicode exception values and unprintable objects - (issue87) fix unboundlocal error in assertionold code - (issue86) improve documentation for looponfailing - refine IO capturing: stdin-redirect pseudo-file now has a NOP close() method - ship distribute_setup.py version 0.6.10 - added links to the new capturelog and coverage plugins From holger at merlinux.eu Wed May 5 21:53:18 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 5 May 2010 21:53:18 +0200 Subject: [py-dev] pytest-xdist 1.2: distributed testing getting stable Message-ID: <20100505195318.GQ17693@trillke.net> pytest-xdist 1.2 brings bug fixes and improved stability for distributed testing with py.test. This plugin allows to ad-hoc distribute test runs to multiple CPUs, different Python interpreters or remote machines. It requires setuptools or distribute which help to pull in the neccessary execnet and pytest-core dependencies. See here for extensive docs: http://pytest.org/plugin/xdist.html best, holger 1.2 ------------------------- - fix issue79: sessionfinish/teardown hooks are now called systematically on the slave side - introduce a new data input/output mechanism to allow the master side to send and receive data from a slave. - fix race condition in underlying pickling/unpickling handling - use and require new register hooks facility of py.test>=1.3.0 - require improved execnet>=1.0.6 because of various race conditions that can arise in xdist testing modes. - fix some python3 related pickling related race conditions - fix PyPI description From adrian.dziubek at gmail.com Fri May 7 11:51:30 2010 From: adrian.dziubek at gmail.com (Adrian Dziubek) Date: Fri, 7 May 2010 11:51:30 +0200 Subject: [py-dev] [py.test] import context problems (causes Django unit test failure) Message-ID: I posted the following on StackOverflow: http://stackoverflow.com/questions/2783090/py-test-import-context-problems-causes-django-unit-test-failure Please, take a look. Thanks in advance! -- Adrian From holger at merlinux.eu Sun May 9 11:25:18 2010 From: holger at merlinux.eu (holger krekel) Date: Sun, 9 May 2010 11:25:18 +0200 Subject: [py-dev] [py.test] import context problems (causes Django unit test failure) In-Reply-To: References: Message-ID: <20100509092518.GT17693@trillke.net> Hi Adrian, For me on both py-1.2.1 and py-1.3.0 things seems to work fine when i type in your example (on a a Mac machine). could you zip up the exact directory? best & greetings from Pycon Italy :) holger On Fri, May 07, 2010 at 11:51 +0200, Adrian Dziubek wrote: > I posted the following on StackOverflow: > > http://stackoverflow.com/questions/2783090/py-test-import-context-problems-causes-django-unit-test-failure > > Please, take a look. Thanks in advance! > -- > Adrian > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From adrian.dziubek at gmail.com Sun May 9 14:49:32 2010 From: adrian.dziubek at gmail.com (Adrian Dziubek) Date: Sun, 9 May 2010 14:49:32 +0200 Subject: [py-dev] [py.test] import context problems (causes Django unit test failure) In-Reply-To: <20100509092518.GT17693@trillke.net> References: <20100509092518.GT17693@trillke.net> Message-ID: 2010/5/9 holger krekel : > Hi Adrian, > > For me on both py-1.2.1 and py-1.3.0 things seems to work fine when > i type in your example (on a a Mac machine). > could you zip up the exact directory? I did the test on my Linux laptop and it works fine too, strange... I'll try again tomorrow at work, where I left the original example. It's probably some setup problem then. Thanks for answering. -- Adrian From adrian.dziubek at gmail.com Mon May 10 13:19:07 2010 From: adrian.dziubek at gmail.com (Adrian Dziubek) Date: Mon, 10 May 2010 13:19:07 +0200 Subject: [py-dev] [py.test] import context problems (causes Django unit test failure) In-Reply-To: References: <20100509092518.GT17693@trillke.net> Message-ID: 2010/5/9 Adrian Dziubek : > 2010/5/9 holger krekel : >> Hi Adrian, >> >> For me on both py-1.2.1 and py-1.3.0 things seems to work fine when >> i type in your example (on a a Mac machine). >> could you zip up the exact directory? > I did the test on my Linux laptop and it works fine too, strange... > I'll try again tomorrow at work, where I left the original example. > It's probably some setup problem then. Thanks for answering. I cannot replicate it any more at work. I found a __init__.py file in moduletest directory, but I can remember explicitly caring about not putting it there... I must have been tired on Friday. I also realised that Django puts __init__.py in the project directory, so it's no wonder, that py.test treats the whole project as a package. I'm still having problems with reverse() inside Django, but it seems irrelevant to py.test. Sorry for bothering, I'll remove the StackOverflow question too. -- Adrian From holger at merlinux.eu Tue May 11 11:19:24 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 11 May 2010 11:19:24 +0200 Subject: [py-dev] [py.test] import context problems (causes Django unit test failure) In-Reply-To: References: <20100509092518.GT17693@trillke.net> Message-ID: <20100511091924.GW17693@trillke.net> On Mon, May 10, 2010 at 13:19 +0200, Adrian Dziubek wrote: > 2010/5/9 Adrian Dziubek : > > 2010/5/9 holger krekel : > >> Hi Adrian, > >> > >> For me on both py-1.2.1 and py-1.3.0 things seems to work fine when > >> i type in your example (on a a Mac machine). > >> could you zip up the exact directory? > > I did the test on my Linux laptop and it works fine too, strange... > > I'll try again tomorrow at work, where I left the original example. > > It's probably some setup problem then. Thanks for answering. > > I cannot replicate it any more at work. I found a __init__.py file in > moduletest directory, but I can remember explicitly caring about not > putting it there... I must have been tired on Friday. I also realised > that Django puts __init__.py in the project directory, so it's no > wonder, that py.test treats the whole project as a package. > > I'm still having problems with reverse() inside Django, but it seems > irrelevant to py.test. > > Sorry for bothering, I'll remove the StackOverflow question too. thanks for getting back on this. Maybe it would help if py.test would be more verbose about "test package root" detection or even let projects define this explicitely. best, holger From simon.callan at infoshare-is.com Tue May 11 11:44:27 2010 From: simon.callan at infoshare-is.com (Simon Callan) Date: Tue, 11 May 2010 10:44:27 +0100 Subject: [py-dev] Setup and teardown functions Message-ID: <001e01caf0ee$8be1bc70$a3a53550$@callan@infoshare-is.com> Hi, I'm trying to understand how to implement setup() and teardown() functions, but finding it a bit difficult. What I want is the following: 1) A setup() teardown() executed when the test process starts and stops. 2) A setup() teardown() executed when a group of tests are started /finished. 3) A setup() / teardown() executed when a test is run. I.e., something like this: global_setup() group1_setup() setup() test_1() teardown() setup() test_2() teardown() setup() test_3() teardown() group1_teardown() group2_setup() setup() test_4() teardown() setup() test_5() teardown() setup() test_6() teardown() group2_teardown() global_teardown() Is this possible? The documentation implies that it should be possible, but I cannot figure it out. Simon Infoshare Ltd Millennium House 21 Eden Street Kingston upon Thames Surrey KT1 1BL United Kingdom Phone: + 44 (0) 20 8541 0111 Support: + 44 (0) 20 8481 4760 Web: www.infoshare-is.com E-mail: info at infoshare-is.com Infoshare Ltd is registered in England and Wales. Registered Office as above. Registered Number 2877612 VAT Number GB 678 1443 10 The content of this e-mail (and any attachment to it) is confidential. Any views or opinions do not represent the views or opinions of Infoshare Ltd. If you have received this e-mail in error please notify the sender and delete it. You may not use, copy or disclose the information in any way. Infoshare Ltd monitors incoming and outgoing e-mails. Please consider the environment. Do you really need to print this email? From holger at merlinux.eu Tue May 11 12:02:24 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 11 May 2010 12:02:24 +0200 Subject: [py-dev] Setup and teardown functions In-Reply-To: <001e01caf0ee$8be1bc70$a3a53550$@callan@infoshare-is.com> References: <001e01caf0ee$8be1bc70$a3a53550$@callan@infoshare-is.com> Message-ID: <20100511100224.GZ17693@trillke.net> Hi Simon, i think i can whip up an example. One question though: Do you already have a view on how you want to to group tests in your source code? In principle you can do this by classes/modules or by decorators - or by having custom ways. cheers, holger On Tue, May 11, 2010 at 10:44 +0100, Simon Callan wrote: > Hi, > > I'm trying to understand how to implement setup() and teardown() functions, > but finding it a bit difficult. > > What I want is the following: > 1) A setup() teardown() executed when the test process starts and stops. > 2) A setup() teardown() executed when a group of tests are started > /finished. > 3) A setup() / teardown() executed when a test is run. > > I.e., something like this: > global_setup() > group1_setup() > setup() > test_1() > teardown() > setup() > test_2() > teardown() > setup() > test_3() > teardown() > group1_teardown() > group2_setup() > setup() > test_4() > teardown() > setup() > test_5() > teardown() > setup() > test_6() > teardown() > group2_teardown() > global_teardown() > > Is this possible? The documentation implies that it should be possible, but > I cannot figure it out. > > Simon > > Infoshare Ltd > Millennium House > 21 Eden Street > Kingston upon Thames > Surrey > KT1 1BL > United Kingdom > > Phone: + 44 (0) 20 8541 0111 > Support: + 44 (0) 20 8481 4760 > Web: www.infoshare-is.com > E-mail: info at infoshare-is.com > > Infoshare Ltd is registered in England and Wales. > Registered Office as above. > Registered Number 2877612 > VAT Number GB 678 1443 10 > > The content of this e-mail (and any attachment to it) is confidential. > Any views or opinions do not represent the views or opinions > of Infoshare Ltd. > If you have received this e-mail in error please notify the sender > and delete it. You may not use, copy or disclose the information > in any way. > > Infoshare Ltd monitors incoming and outgoing e-mails. > > Please consider the environment. Do you really need to print > this email? > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From simon.callan at infoshare-is.com Tue May 11 12:32:30 2010 From: simon.callan at infoshare-is.com (Simon Callan) Date: Tue, 11 May 2010 11:32:30 +0100 Subject: [py-dev] Setup and teardown functions In-Reply-To: <20100511100224.GZ17693@trillke.net> References: <001e01caf0ee$8be1bc70$a3a53550$@callan@infoshare-is.com> <20100511100224.GZ17693@trillke.net> Message-ID: <001f01caf0f5$426bb850$c74328f0$@callan@infoshare-is.com> Hi, > i think i can whip up an example. One question though: Do you already have a > view on how you want to to group tests in your source code? In principle you > can do this by classes/modules or by decorators - or by having custom ways. We're still at the early stages of laying out our tests, so we're pretty free-form at the moment, however, I think the easiest way to do it would be to have all the test functions in a module to be considered part of a group. Simon Infoshare Ltd Millennium House 21 Eden Street Kingston upon Thames Surrey KT1 1BL United Kingdom Phone: + 44 (0) 20 8541 0111 Support: + 44 (0) 20 8481 4760 Web: www.infoshare-is.com E-mail: info at infoshare-is.com Infoshare Ltd is registered in England and Wales. Registered Office as above. Registered Number 2877612 VAT Number GB 678 1443 10 The content of this e-mail (and any attachment to it) is confidential. Any views or opinions do not represent the views or opinions of Infoshare Ltd. If you have received this e-mail in error please notify the sender and delete it. You may not use, copy or disclose the information in any way. Infoshare Ltd monitors incoming and outgoing e-mails. Please consider the environment. Do you really need to print this email? From holger at merlinux.eu Tue May 11 15:33:07 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 11 May 2010 15:33:07 +0200 Subject: [py-dev] Setup and teardown functions In-Reply-To: <001f01caf0f5$426bb850$c74328f0$@callan@infoshare-is.com> References: <20100511100224.GZ17693@trillke.net> <001f01caf0f5$426bb850$c74328f0$@callan@infoshare-is.com> Message-ID: <20100511133306.GA17693@trillke.net> Hi Simon, i attach an initial solution as a tar.gz file: * tests/conftest.py : the per-project file to put your configuration and test setup machinery in * tests/test_module1.py a test module with a test function and a test method * tests/test_module2.py another test module for demo purposes If you untar and run "py.test", a 'setup.log' file gets created which visualizes setup and teardown invocations: global_setup() group_setup('tests.test_module1') setup() executing function 'test_function' teardown() setup() executing method 'test_method' teardown() group_teardown('tests.test_module1') group_setup('tests.test_module2') setup() executing function 'test_function' teardown() setup() executing method 'test_method' teardown() group_teardown('tests.test_module2') global_teardown() A few notes which you might find helpful for understanding the solution: * each test functions needs to note down the 'mysetup' argument. It gives each test function to global/group/per-test function setup state in case you acually do anything in the setup/teardown functions other than logging and want the tests to access it :) If you don't specify the 'mysetup' argument your tests will still run but not trigger any setup/teardown functions - might be useful for pure unit-tests that don't depend on prepared state. * funcargs, the request object and the cached_setup() method are documented in http://pytest.org/funcargs.html * you may choose to optionally call "inlined" setup/teardown methods, i.e. ones that are defined in the test module; this can be done easily because the setup functions in the conftest.py file have a reference to the module object (request.module) or test functions (request.function). You may alternatively choose to declare test configuration options in a test module or per decorators to test functions and use this to drive the setup/teardown machinery, i.e. put into a test module a variable like: group_config = dict(key1=value1, ...) and interpret it from the hooks in the conftest.py. And maybe a final note: the mechanism i noted down requires conftest.py writers to understand py.test internals but test module writers can remain ignorant about it. Please let me know if this makes sense to you. best, holger On Tue, May 11, 2010 at 11:32 +0100, Simon Callan wrote: > Hi, > > > i think i can whip up an example. One question though: Do you already > have a > > view on how you want to to group tests in your source code? In principle > you > > can do this by classes/modules or by decorators - or by having custom > ways. > > We're still at the early stages of laying out our tests, so we're pretty > free-form at the moment, however, I think the easiest way to do it would be > to have all the test functions in a module to be considered part of a group. > > Simon > > Infoshare Ltd > Millennium House > 21 Eden Street > Kingston upon Thames > Surrey > KT1 1BL > United Kingdom > > Phone: + 44 (0) 20 8541 0111 > Support: + 44 (0) 20 8481 4760 > Web: www.infoshare-is.com > E-mail: info at infoshare-is.com > > Infoshare Ltd is registered in England and Wales. > Registered Office as above. > Registered Number 2877612 > VAT Number GB 678 1443 10 > > The content of this e-mail (and any attachment to it) is confidential. > Any views or opinions do not represent the views or opinions > of Infoshare Ltd. > If you have received this e-mail in error please notify the sender > and delete it. You may not use, copy or disclose the information > in any way. > > Infoshare Ltd monitors incoming and outgoing e-mails. > > Please consider the environment. Do you really need to print > this email? > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- -------------- next part -------------- A non-text attachment was scrubbed... Name: setup-example.tar.gz Type: application/octet-stream Size: 722 bytes Desc: not available URL: From simon.callan at infoshare-is.com Tue May 11 18:38:32 2010 From: simon.callan at infoshare-is.com (Simon Callan) Date: Tue, 11 May 2010 17:38:32 +0100 Subject: [py-dev] Setup and teardown functions In-Reply-To: <20100511133306.GA17693@trillke.net> References: <20100511100224.GZ17693@trillke.net> <001f01caf0f5$426bb850$c74328f0$@callan@infoshare-is.com> <20100511133306.GA17693@trillke.net> Message-ID: <004801caf128$64b638d0$2e22aa70$@callan@infoshare-is.com> Hi, > i attach an initial solution as a tar.gz file: > > * tests/conftest.py : the per-project file to put your configuration > and test setup machinery in > * tests/test_module1.py a test module with a test function and a test method > * tests/test_module2.py another test module for demo purposes ... > Please let me know if this makes sense to you. This appears to be just what I wanted, and what I was slowly groping toward myself. I now have another question. Normally, py.test is started using the executable. Is it possible to start it from a python script? This would make it a lot easier to debug the test scripts, as I could set breakpoints, etc using PyDev. Simon Infoshare Ltd Millennium House 21 Eden Street Kingston upon Thames Surrey KT1 1BL United Kingdom Phone: + 44 (0) 20 8541 0111 Support: + 44 (0) 20 8481 4760 Web: www.infoshare-is.com E-mail: info at infoshare-is.com Infoshare Ltd is registered in England and Wales. Registered Office as above. Registered Number 2877612 VAT Number GB 678 1443 10 The content of this e-mail (and any attachment to it) is confidential. Any views or opinions do not represent the views or opinions of Infoshare Ltd. If you have received this e-mail in error please notify the sender and delete it. You may not use, copy or disclose the information in any way. Infoshare Ltd monitors incoming and outgoing e-mails. Please consider the environment. Do you really need to print this email? From sdoherty at avvasi.com Tue May 11 19:30:37 2010 From: sdoherty at avvasi.com (Stuart Doherty) Date: Tue, 11 May 2010 13:30:37 -0400 Subject: [py-dev] Setup and teardown functions In-Reply-To: <004801caf128$64b638d0$2e22aa70$@callan@infoshare-is.com> References: <20100511100224.GZ17693@trillke.net> <001f01caf0f5$426bb850$c74328f0$@callan@infoshare-is.com> <20100511133306.GA17693@trillke.net>, <004801caf128$64b638d0$2e22aa70$@callan@infoshare-is.com> Message-ID: <02AF59B4F3B6F242A5F08CCACBBF9BD4087C667FF2@air.MATTER.LOCAL> Hi Simon, I'm doing just that by importing the "py" package, and invoking py.test with: >>>py.test.cmdline.main([]]) The argument to the "main" function is a python list of the arguments you would pass at the shell prompt. I've been able to debug using pydev/eclipse. Stu ________________________________________ From: py-dev-bounces at codespeak.net [py-dev-bounces at codespeak.net] On Behalf Of Simon Callan [simon.callan at infoshare-is.com] Sent: May 11, 2010 12:38 PM To: py-dev at codespeak.net Subject: Re: [py-dev] Setup and teardown functions Hi, > i attach an initial solution as a tar.gz file: > > * tests/conftest.py : the per-project file to put your configuration > and test setup machinery in > * tests/test_module1.py a test module with a test function and a test method > * tests/test_module2.py another test module for demo purposes ... > Please let me know if this makes sense to you. This appears to be just what I wanted, and what I was slowly groping toward myself. I now have another question. Normally, py.test is started using the executable. Is it possible to start it from a python script? This would make it a lot easier to debug the test scripts, as I could set breakpoints, etc using PyDev. Simon Infoshare Ltd Millennium House 21 Eden Street Kingston upon Thames Surrey KT1 1BL United Kingdom Phone: + 44 (0) 20 8541 0111 Support: + 44 (0) 20 8481 4760 Web: www.infoshare-is.com E-mail: info at infoshare-is.com Infoshare Ltd is registered in England and Wales. Registered Office as above. Registered Number 2877612 VAT Number GB 678 1443 10 The content of this e-mail (and any attachment to it) is confidential. Any views or opinions do not represent the views or opinions of Infoshare Ltd. If you have received this e-mail in error please notify the sender and delete it. You may not use, copy or disclose the information in any way. Infoshare Ltd monitors incoming and outgoing e-mails. Please consider the environment. Do you really need to print this email? _______________________________________________ py-dev mailing list py-dev at codespeak.net http://codespeak.net/mailman/listinfo/py-dev From holger at merlinux.eu Tue May 11 19:56:29 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 11 May 2010 19:56:29 +0200 Subject: [py-dev] Setup and teardown functions In-Reply-To: <004801caf128$64b638d0$2e22aa70$@callan@infoshare-is.com> References: <20100511100224.GZ17693@trillke.net> <20100511133306.GA17693@trillke.net> <004801caf128$64b638d0$2e22aa70$@callan@infoshare-is.com> Message-ID: <20100511175629.GC17693@trillke.net> On Tue, May 11, 2010 at 17:38 +0100, Simon Callan wrote: > > i attach an initial solution as a tar.gz file: > > > > * tests/conftest.py : the per-project file to put your configuration > > and test setup machinery in > > * tests/test_module1.py a test module with a test function and a test > method > > * tests/test_module2.py another test module for demo purposes > ... > > Please let me know if this makes sense to you. > > This appears to be just what I wanted, and what I was slowly groping toward > myself. > > I now have another question. > > Normally, py.test is started using the executable. Is it possible to start > it from a python script? This would make it a lot easier to debug the test > scripts, as I could set breakpoints, etc using PyDev. In principle you can put this in a script: import py py.test.cmdline.main() or if you want to repeatedly run tests in the same process you may use this helper function: https://bitbucket.org/hpk42/py-trunk/src/tip/contrib/runtesthelper.py I also just committed a fix that makes the runtesthelper script unneccessary - so in >=1.3.1 you will just be able to call py.test.cmdline.main() multiple times in the same process. best, holger From issues-noreply at bitbucket.org Wed May 12 11:14:14 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Wed, 12 May 2010 09:14:14 -0000 Subject: [py-dev] New issue 93 in py-trunk: stdout should be captured while importing conftest.py Message-ID: <482ec0488b4e01c046cef853fb32f476@bitbucket.org> --- you can reply above this line --- New issue 93: stdout should be captured while importing conftest.py http://bitbucket.org/hpk42/py-trunk/issue/93/stdout-should-be-captured-while-importing amauryfa on Wed, 12 May 2010 11:14:14 +0200: Description: With py.test 1.1.1, conftest.py was imported during the collection phase, and stdout was in 'captured' mode. But with 1.3.0, stdout is not captured any more. This makes a difference in pypy, where importing some package have side-effects (configuration, compilation attempts, etc.) that prints several lines. These lines should be masked, and output captured, like it is in setup_module(), for example. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Wed May 12 11:59:49 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Wed, 12 May 2010 09:59:49 -0000 Subject: [py-dev] New issue 94 in py-trunk: py.test can stop with no output Message-ID: <783856a912d1e21a209823faaad927b7@bitbucket.org> --- you can reply above this line --- New issue 94: py.test can stop with no output http://bitbucket.org/hpk42/py-trunk/issue/94/pytest-can-stop-with-no-output amauryfa on Wed, 12 May 2010 11:59:48 +0200: Description: To reproduce: in current pypy (r74497), add a new AppTest: {{{ #!python class AppTest: def test_raises(self): raises(ValueError, "x") }}} You may have to run the test twice to get the problem. This triggers three bugs in various places: * pypy uses __file__ to create traceback entries, which may end in '.pyc' (the normal way is to use co_filename, which is always a '.py') * then py.test attempts to get the source code, but opens the .pyc file, and calls syntax_checker(). This fails of course and leads to a pile of INTERNALERROR lines. * then pytest_capture is lost: pytest_runtest_makereport failed, suspendcapture() is not called, and the next call to resumecapture() will raise the error "cannot resume, already capturing with 'no'". I think all three places should be modified (and it's not only pypy's fault!) Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Fri May 14 12:05:44 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Fri, 14 May 2010 10:05:44 -0000 Subject: [py-dev] New issue 95 in py-trunk: py.test will not start if zlib is not available Message-ID: --- you can reply above this line --- New issue 95: py.test will not start if zlib is not available http://bitbucket.org/hpk42/py-trunk/issue/95/pytest-will-not-start-if-zlib-is-not amauryfa on Fri, 14 May 2010 12:05:43 +0200: Description: if the zlib module is not available (because it's a custom build of python or pypy), py.test will fail to import the pytest_genscript plugin, and nothing will work. if zlib is not available, the "--genscript" option cannot work, but this should not prevent py.test from running tests. I suggest a line like the following: {{{ zlib = py.test.importorskip("zlib") }}} Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Sun May 16 23:30:31 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Sun, 16 May 2010 21:30:31 -0000 Subject: [py-dev] New issue 96 in py-trunk: AssertionError when cancelling a test Message-ID: --- you can reply above this line --- New issue 96: AssertionError when cancelling a test http://bitbucket.org/hpk42/py-trunk/issue/96/assertionerror-when-cancelling-a-test amauryfa on Sun, 16 May 2010 23:30:30 +0200: Description: I got this traceback when pressing Ctrl-C. This happens from time to time, and there is not line "!!! KEYBOARD INTERRUPT !!!" {{{ module\cpyext\test\test_typeobject.py .......... module\cpyext\test\test_unicodeobject.py Traceback (most recent call last): File "C:\dev\pypy\trunk\pypy\test_all.py", line 6, in ? py.cmdline.pytest() File "C:\dev\pypy\trunk\py\_test\cmdline.py", line 17, in main exitstatus = session.main(colitems) File "C:\dev\pypy\trunk\py\_test\session.py", line 113, in main self.config.hook.pytest_keyboard_interrupt(excinfo=excinfo) File "C:\dev\pypy\trunk\py\_test\pluginmanager.py", line 347, in __call__ return self.hookrelay._performcall(self.name, mc) File "C:\dev\pypy\trunk\py\_test\pluginmanager.py", line 333, in _performcall return multicall.execute() File "C:\dev\pypy\trunk\py\_test\pluginmanager.py", line 244, in execute res = method(**kwargs) File "C:\dev\pypy\trunk\py\_plugin\pytest_capture.py", line 232, in pytest_keyboard_interrupt self.suspendcapture() File "C:\dev\pypy\trunk\py\_plugin\pytest_capture.py", line 170, in suspendcapture outerr = cap.suspend() File "C:\dev\pypy\trunk\py\_io\capture.py", line 166, in suspend assert not hasattr(self, '_suspended') py._code.assertion.AssertionError: assert not hasattr(, '_suspended') }}} Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Mon May 17 11:59:40 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Mon, 17 May 2010 09:59:40 -0000 Subject: [py-dev] New issue 97 in py-trunk: Dynamic distribution of tests on different hosts Message-ID: <8edcd9abd0a7c415b0bcf9de364b295b@bitbucket.org> --- you can reply above this line --- New issue 97: Dynamic distribution of tests on different hosts http://bitbucket.org/hpk42/py-trunk/issue/97/dynamic-distribution-of-tests-on-different Noufal Ibrahim / nibrahim on Mon, 17 May 2010 11:59:40 +0200: Description: Right now, the host setup is done globally upfront. This means that an invocation of py.test can send tests only to a single set of hosts. It would be nice to be able to customise this in the `conftest.py` files so that I can indicate that all tests in (say) "test_foo" directory should be run on host X. The person who invokes the tests wouldn't need to care and can directly just run tests. The config options would take care of distributing the tests to the right machines. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Mon May 17 12:02:29 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Mon, 17 May 2010 10:02:29 -0000 Subject: [py-dev] New issue 98 in py-trunk: Option errors in xdist documentation Message-ID: <7680533c0ce34e05b2a0e6e243321429@bitbucket.org> --- you can reply above this line --- New issue 98: Option errors in xdist documentation http://bitbucket.org/hpk42/py-trunk/issue/98/option-errors-in-xdist-documentation Noufal Ibrahim / nibrahim on Mon, 17 May 2010 12:02:29 +0200: Description: Using some pytest_option settings as described in http://codespeak.net/py/dist/test/plugin/xdist.html#specifying-test-exec-environments-in-a-conftest-py results in a py._plugin.pytest_helpconfig.PluginValidationError: being raised with a "found unknown hook: pytest_option_tx" message. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From simon.callan at infoshare-is.com Mon May 17 17:27:24 2010 From: simon.callan at infoshare-is.com (Simon Callan) Date: Mon, 17 May 2010 16:27:24 +0100 Subject: [py-dev] Running the least run test Message-ID: <002501caf5d5$73181e00$59485a00$@callan@infoshare-is.com> Hi, Having got good answers to my previous questions, I'm back for more help, with a few related questions. 1) Is it possible to programmatically stop py.test halfway through running a set of tests? I.e., for an external program to say "complete the current test and then stop". 2) If I have a number of tests (let's say a1 to a5), and I aborted the test during the running of test a3, is it possible for the next invocation of py.test to remember where it got to, and start running again at test a4? Simon Infoshare Ltd Millennium House 21 Eden Street Kingston upon Thames Surrey KT1 1BL United Kingdom Phone: + 44 (0) 20 8541 0111 Support: + 44 (0) 20 8481 4760 Web: www.infoshare-is.com E-mail: info at infoshare-is.com Infoshare Ltd is registered in England and Wales. Registered Office as above. Registered Number 2877612 VAT Number GB 678 1443 10 The content of this e-mail (and any attachment to it) is confidential. Any views or opinions do not represent the views or opinions of Infoshare Ltd. If you have received this e-mail in error please notify the sender and delete it. You may not use, copy or disclose the information in any way. Infoshare Ltd monitors incoming and outgoing e-mails. Please consider the environment. Do you really need to print this email? From holger at merlinux.eu Mon May 17 19:18:20 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 17 May 2010 19:18:20 +0200 Subject: [py-dev] Running the least run test In-Reply-To: <002501caf5d5$73181e00$59485a00$@callan@infoshare-is.com> References: <002501caf5d5$73181e00$59485a00$@callan@infoshare-is.com> Message-ID: <20100517171820.GN17693@trillke.net> Hi Simon, On Mon, May 17, 2010 at 16:27 +0100, Simon Callan wrote: > Having got good answers to my previous questions, I'm back for more help, > with a few related questions. > > 1) Is it possible to programmatically stop py.test halfway through running a > set of tests? I.e., for an external program to say "complete the current > test and then stop". That's not easily possible although distributed testing internally this feature. Can you create an enhancement issue and maybe describe your use case / intention of using it a bit more? This helps me a lot to avoid coding features nobody needs. > 2) If I have a number of tests (let's say a1 to a5), and I aborted the test > during the running of test a3, is it possible for the next invocation of > py.test to remember where it got to, and start running again at test a4? Rerunning the last failure(s) and continuing on would probably be nice. It's not possible currently, requires adding some persistence between py.test invocations. Could you also create an issue? If you or somebody else has anything to add or to suggest, shoot :) holger From meine at informatik.uni-hamburg.de Thu May 20 12:43:13 2010 From: meine at informatik.uni-hamburg.de (Hans Meine) Date: Thu, 20 May 2010 12:43:13 +0200 Subject: [py-dev] Problems with test discovery (ImportError) Message-ID: <201005201243.13849.meine@informatik.uni-hamburg.de> Hi, I had successfully toyed with py.test in the past, and just wanted to use it again, instead of nosetests. However, py.test fails with E ImportError: No module named something.test_foo This seems to be caused by a package subdirectory called 'io'. Here's how to reproduce this (not necessarily minimal yet): mkdir -p test_ImportError/io/something touch test_ImportError/io/something/test_foo.py touch test_ImportError/io/something/__init__.py touch test_ImportError/io/__init__.py py.test test_ImportError/io/something Obviously, this is a bug that should be fixed, but I would already happy if you could tell me a workaround (i.e. different invocation of py.test, without changing the directory name of course). Have a nice day, Hans PS: Please CC me as I did not subscribe for this single report. Thanks. From holger at merlinux.eu Thu May 20 12:49:31 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 20 May 2010 12:49:31 +0200 Subject: [py-dev] Problems with test discovery (ImportError) In-Reply-To: <201005201243.13849.meine@informatik.uni-hamburg.de> References: <201005201243.13849.meine@informatik.uni-hamburg.de> Message-ID: <20100520104931.GV17693@trillke.net> Hi Hans, On Thu, May 20, 2010 at 12:43 +0200, Hans Meine wrote: > Hi, > > I had successfully toyed with py.test in the past, and just wanted to use it > again, instead of nosetests. However, py.test fails with > > E ImportError: No module named something.test_foo > > This seems to be caused by a package subdirectory called 'io'. Here's how to > reproduce this (not necessarily minimal yet): > > mkdir -p test_ImportError/io/something > touch test_ImportError/io/something/test_foo.py > touch test_ImportError/io/something/__init__.py > touch test_ImportError/io/__init__.py > py.test test_ImportError/io/something > > Obviously, this is a bug that should be fixed, but I would already happy if > you could tell me a workaround (i.e. different invocation of py.test, without > changing the directory name of course). The error message is a bit obscure but scrolling a bit up you see that py.test tries to import 'io.something.test_foo' - i.e. you have created a package 'io' and that overlaps with the standard library's 'io' package. Solution: rename 'io' to something else (and generally don't use a package name that overlaps with standard library ones). cheers, holger From meine at informatik.uni-hamburg.de Thu May 20 13:12:16 2010 From: meine at informatik.uni-hamburg.de (Hans Meine) Date: Thu, 20 May 2010 13:12:16 +0200 Subject: [py-dev] Problems with test discovery (ImportError) In-Reply-To: <20100520104931.GV17693@trillke.net> References: <201005201243.13849.meine@informatik.uni-hamburg.de> <20100520104931.GV17693@trillke.net> Message-ID: <201005201312.16344.meine@informatik.uni-hamburg.de> On Thursday 20 May 2010 12:49:31 holger krekel wrote: > On Thu, May 20, 2010 at 12:43 +0200, Hans Meine wrote: > > [...] py.test fails with > > > > E ImportError: No module named something.test_foo > > > > [...] > > > > Obviously, this is a bug that should be fixed, but I would already happy > > if you could tell me a workaround (i.e. different invocation of py.test, > > without changing the directory name of course). > > The error message is a bit obscure but scrolling a bit up you see > that py.test tries to import 'io.something.test_foo' Can I circumvent this? (IIRC, for instance nosetests differentiates between full filenames and importable module names.) > - i.e. you have created a package 'io' and that overlaps with the standard > library's 'io' package. Ah, bad. But I won't change that now, after using the name myself for several years already. Who needs stdlib's io anyway.. ;-p Another workaround is to prepend my main package directory to PYTHONPATH (tested, works -> fine). Thanks for the explanation, Hans PS: No need to CC me anymore, the list turned out to be subscribers-only. :-/ From holger at merlinux.eu Thu May 20 14:00:29 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 20 May 2010 14:00:29 +0200 Subject: [py-dev] Problems with test discovery (ImportError) In-Reply-To: <201005201312.16344.meine@informatik.uni-hamburg.de> References: <201005201243.13849.meine@informatik.uni-hamburg.de> <20100520104931.GV17693@trillke.net> <201005201312.16344.meine@informatik.uni-hamburg.de> Message-ID: <20100520120029.GW17693@trillke.net> On Thu, May 20, 2010 at 13:12 +0200, Hans Meine wrote: > On Thursday 20 May 2010 12:49:31 holger krekel wrote: > > On Thu, May 20, 2010 at 12:43 +0200, Hans Meine wrote: > > > [...] py.test fails with > > > > > > E ImportError: No module named something.test_foo > > > > > > [...] > > > > > > Obviously, this is a bug that should be fixed, but I would already happy > > > if you could tell me a workaround (i.e. different invocation of py.test, > > > without changing the directory name of course). > > > > The error message is a bit obscure but scrolling a bit up you see > > that py.test tries to import 'io.something.test_foo' > > Can I circumvent this? (IIRC, for instance nosetests differentiates between > full filenames and importable module names.) Could you elaborate on nose's logic a bit? > > - i.e. you have created a package 'io' and that overlaps with the standard > > library's 'io' package. > > Ah, bad. But I won't change that now, after using the name myself for several > years already. Who needs stdlib's io anyway.. ;-p Hehe :) However, it only works as long as no pluguin and no code you are using depends on the standard 'io' module. py.test itself uses a fallback in case it can't "from io import StringIO" but in general a user package named 'io' will lead to breakage, latest when using Python3. > Another workaround is to prepend my main package directory to PYTHONPATH > (tested, works -> fine). You could also try to not have __init__.py files in test directories - as long as all tests have unique names that's fine (and py.test should warn you if there is a conflict). > Thanks for the explanation, > Hans have fun, holger > PS: No need to CC me anymore, the list turned out to be subscribers-only. :-/ P.S.: yeah, sorry. otherwise a lot of spam-sorting would be needed to let the occassional post through. From anto.cuni at gmail.com Thu May 20 14:01:49 2010 From: anto.cuni at gmail.com (Antonio Cuni) Date: Thu, 20 May 2010 14:01:49 +0200 Subject: [py-dev] Running the least run test In-Reply-To: <20100517171820.GN17693@trillke.net> References: <002501caf5d5$73181e00$59485a00$@callan@infoshare-is.com> <20100517171820.GN17693@trillke.net> Message-ID: <4BF524AD.4080308@gmail.com> On 17/05/10 19:18, holger krekel wrote: >> 2) If I have a number of tests (let's say a1 to a5), and I aborted the test >> during the running of test a3, is it possible for the next invocation of >> py.test to remember where it got to, and start running again at test a4? > > Rerunning the last failure(s) and continuing on would probably be nice. > It's not possible currently, requires adding some persistence > between py.test invocations. Could you also create an issue? > > If you or somebody else has anything to add or to suggest, shoot :) some time ago I implemented such a feature without any need to add persistence: basically, I assigned a progressive number to each test (printed on failure) and added an option to skip the first N. In combination with -x it did exactly what Simon described, but unfortunately I lost the code during an svn up. I think it can be a good quick&dirty solution, while waiting for the proper one with persistence between runs. ciao, Anto From issues-noreply at bitbucket.org Fri May 21 04:20:55 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Fri, 21 May 2010 02:20:55 -0000 Subject: [py-dev] New issue 99 in py-trunk: unexpected pass crashes with junitxml plugin Message-ID: <2b257e921a78f655e5186375dd4a5404@bitbucket.org> --- you can reply above this line --- New issue 99: unexpected pass crashes with junitxml plugin http://bitbucket.org/hpk42/py-trunk/issue/99/unexpected-pass-crashes-with-junitxml Anonymous on Fri, 21 May 2010 04:20:55 +0200: Description: Unexpected pass should cause the tests to report failure, however with the junitxml plugin it crashes with the following: INTERNALERROR> def append_failure(self, report): INTERNALERROR> self._opentestcase(report) INTERNALERROR> #msg = str(report.longrepr.reprtraceback.extraline) INTERNALERROR> self.appendlog('%s', INTERNALERROR> > report.longrepr) INTERNALERROR> E AttributeError: 'ItemTestReport' object has no attribute 'longrepr' -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Tue May 25 11:52:39 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Tue, 25 May 2010 09:52:39 -0000 Subject: [py-dev] New issue 100 in py-trunk: Please tag pytest-xdist 1.2 Message-ID: <028421ae674c296bdec738348c449554@bitbucket.org> --- you can reply above this line --- New issue 100: Please tag pytest-xdist 1.2 http://bitbucket.org/hpk42/py-trunk/issue/100/please-tag-pytest-xdist-12 memedough on Tue, 25 May 2010 11:52:39 +0200: Description: Hi, The pytest-xdist mercurial repo has no tag for release 1.2 which is on pypi. Please add tag. :) Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Tue May 25 12:16:59 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Tue, 25 May 2010 10:16:59 -0000 Subject: [py-dev] New issue 101 in py-trunk: Add pytest-cov to plugins page? Message-ID: <1b120fc54ab73ca097eae5314e4d9af3@bitbucket.org> --- you can reply above this line --- New issue 101: Add pytest-cov to plugins page? http://bitbucket.org/hpk42/py-trunk/issue/101/add-pytest-cov-to-plugins-page memedough on Tue, 25 May 2010 12:16:58 +0200: Description: Hi, Would you be agreeable to adding pytest-cov to the plugins page? I've been using it for a while now and it's performing well and stable for my usage pattern. It provides another option and may be useful for others. :) Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Tue May 25 12:29:30 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Tue, 25 May 2010 10:29:30 -0000 Subject: [py-dev] New issue 102 in py-trunk: Allow -x to take an argument Message-ID: <7a3955868b24eada98145157adb0ed8f@bitbucket.org> --- you can reply above this line --- New issue 102: Allow -x to take an argument http://bitbucket.org/hpk42/py-trunk/issue/102/allow-x-to-take-an-argument alex_gaynor on Tue, 25 May 2010 12:29:29 +0200: Description: Right now -x doesn't take an arguments, and it just stops the tests after a single failure. Sometimes I want to see a few failures, but not all 100 :) If -x had an argument it would basically run the tests up until that number of errors/failures. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From holger at merlinux.eu Tue May 25 12:32:18 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 25 May 2010 12:32:18 +0200 Subject: [py-dev] New issue 101 in py-trunk: Add pytest-cov to plugins page? In-Reply-To: <1b120fc54ab73ca097eae5314e4d9af3@bitbucket.org> References: <1b120fc54ab73ca097eae5314e4d9af3@bitbucket.org> Message-ID: <20100525103218.GL17693@trillke.net> Hi Meme, well, i'd rather like to see the two coverage approaches / features merge into a single good coverage plugin with tests. For my use cases both don't work completely well ;) holger On Tue, May 25, 2010 at 10:16 -0000, issues-noreply at bitbucket.org wrote: > --- you can reply above this line --- > > New issue 101: Add pytest-cov to plugins page? > http://bitbucket.org/hpk42/py-trunk/issue/101/add-pytest-cov-to-plugins-page > > memedough on Tue, 25 May 2010 12:16:58 +0200: > > Description: > Hi, > > Would you be agreeable to adding pytest-cov to the plugins page? I've been using it for a while now and it's performing well and stable for my usage pattern. It provides another option and may be useful for others. > > :) > > > Responsible: > hpk42 > -- > This is an issue notification from bitbucket.org. > You are receiving this either because you are the > owner of the issue, or you are following the issue. > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From holger at merlinux.eu Tue May 25 21:15:28 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 25 May 2010 21:15:28 +0200 Subject: [py-dev] py.test-1.3.1: new py.test.xfail, --maxfail, better reporting, fixes Message-ID: <20100525191528.GM17693@trillke.net> The pylib/py.test 1.3.1 release brings: - the new imperative ``py.test.xfail()`` helper in order to have a test or setup function result in an "expected failure" - a new option ``--maxfail=NUM`` to stop the test run after some failures - markers/decorators are now applicable to test classes (>=Python2.6) - improved reporting, shorter tracebacks in several cases - some simplified internals, more compatibility with Jython and PyPy - bug fixes and various refinements See the below CHANGELOG entry below for more details and http://pylib.org/install.html for installation instructions. If you used older versions of py.test you should be able to upgrade to 1.3.1 without changes to your test source code. py.test is an automated testing tool working with Python2, Python3, Jython and PyPy versions on all major operating systems. It offers a no-boilerplate testing approach and has inspired other testing tools and enhancements in the standard Python library for more than five years. It has a simple and extensive plugin architecture, configurable reporting and provides unique ways to make it fit to your testing process and needs. See http://pytest.org for more info. cheers and have fun, holger krekel Changes between 1.3.0 and 1.3.1 ================================================== New features ++++++++++++++++++ - issue91: introduce new py.test.xfail(reason) helper to imperatively mark a test as expected to fail. Can be used from within setup and test functions. This is useful especially for parametrized tests when certain configurations are expected-to-fail. In this case the declarative approach with the @py.test.mark.xfail cannot be used as it would mark all configurations as xfail. - issue102: introduce new --maxfail=NUM option to stop test runs after NUM failures. This is a generalization of the '-x' or '--exitfirst' option which is now equivalent to '--maxfail=1'. Both '-x' and '--maxfail' will now also print a line near the end indicating the Interruption. - issue89: allow py.test.mark decorators to be used on classes (class decorators were introduced with python2.6) and also allow to have multiple markers applied at class/module level by specifying a list. - improve and refine letter reporting in the progress bar: . pass f failed test s skipped tests (reminder: use for dependency/platform mismatch only) x xfailed test (test that was expected to fail) X xpassed test (test that was expected to fail but passed) You can use any combination of 'fsxX' with the '-r' extended reporting option. The xfail/xpass results will show up as skipped tests in the junitxml output - which also fixes issue99. - make py.test.cmdline.main() return the exitstatus instead of raising SystemExit and also allow it to be called multiple times. This of course requires that your application and tests are properly teared down and don't have global state. Fixes / Maintenance ++++++++++++++++++++++ - improved traceback presentation: - improved and unified reporting for "--tb=short" option - Errors during test module imports are much shorter, (using --tb=short style) - raises shows shorter more relevant tracebacks - --fulltrace now more systematically makes traces longer / inhibits cutting - improve support for raises and other dynamically compiled code by manipulating python's linecache.cache instead of the previous rather hacky way of creating custom code objects. This makes it seemlessly work on Jython and PyPy where it previously didn't. - fix issue96: make capturing more resilient against Control-C interruptions (involved somewhat substantial refactoring to the underlying capturing functionality to avoid race conditions). - fix chaining of conditional skipif/xfail decorators - so it works now as expected to use multiple @py.test.mark.skipif(condition) decorators, including specific reporting which of the conditions lead to skipping. - fix issue95: late-import zlib so that it's not required for general py.test startup. - fix issue94: make reporting more robust against bogus source code (and internally be more careful when presenting unexpected byte sequences) From holger at merlinux.eu Tue May 25 21:35:43 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 25 May 2010 21:35:43 +0200 Subject: [py-dev] pytest-xdist-1.3 fixes and a sidenote/tip Message-ID: <20100525193543.GN17693@trillke.net> Hi again, i also uploaded a new pytest-xdist-1.3 plugin to the package index which requires the new py-1.3.1. Main thing: If you experienced the strange phenomenon of -f|--looponfail to show tracebacks with _old_ source code, this release should fix that. It also adjusts to the new --maxfail=NUM option of py.test so that in a distributed test run this option works properly: aborting the test run if more then the maximum number of failures are encountered, bringing all nodes down. Btw, did you know you can set this as well as other options through your shell environment or a conftest.py file? For example, export PYTEST_OPTION_MAXFAIL=3 # in bash or option_maxfail = 3 # in conftest.py to always abort test runs after three failures. To find out about available ENV or conftest.py names type py.test --help-config best, holger From holger at merlinux.eu Wed May 26 14:57:01 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 26 May 2010 14:57:01 +0200 Subject: [py-dev] coverage (was issue101) In-Reply-To: <20100525103218.GL17693@trillke.net> References: <1b120fc54ab73ca097eae5314e4d9af3@bitbucket.org> <20100525103218.GL17693@trillke.net> Message-ID: <20100526125701.GQ17693@trillke.net> Hi Meme, i actually now added your pytest-cov plugin to the plugin index page: http://pytest.org/plugin/index.html :) We now advertise two coverage plugins, not a too fortunate situation. seems like your plugin now works much better for me than some versions ago. I still find your command line option design a bit obnoxious - i usually try to keep it much more minimal. Are you open to advise/suggestions there? Also, is it possible to start a Python script/subprocess and get its coverage included as well? pylib/.test itself has a lot of functional tests and those start subprocesses which is a usual scenario. best, holger On Tue, May 25, 2010 at 12:32 +0200, holger krekel wrote: > Hi Meme, > > well, i'd rather like to see the two coverage approaches / features merge > into a single good coverage plugin with tests. For my use cases > both don't work completely well ;) > > holger > > On Tue, May 25, 2010 at 10:16 -0000, issues-noreply at bitbucket.org wrote: > > > --- you can reply above this line --- > > > > New issue 101: Add pytest-cov to plugins page? > > http://bitbucket.org/hpk42/py-trunk/issue/101/add-pytest-cov-to-plugins-page > > > > memedough on Tue, 25 May 2010 12:16:58 +0200: > > > > Description: > > Hi, > > > > Would you be agreeable to adding pytest-cov to the plugins page? I've been using it for a while now and it's performing well and stable for my usage pattern. It provides another option and may be useful for others. > > > > :) > > > > > > Responsible: > > hpk42 > > -- > > This is an issue notification from bitbucket.org. > > You are receiving this either because you are the > > owner of the issue, or you are following the issue. > > _______________________________________________ > > py-dev mailing list > > py-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/py-dev > > > > -- > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From memedough at gmail.com Thu May 27 13:24:55 2010 From: memedough at gmail.com (meme dough) Date: Thu, 27 May 2010 21:24:55 +1000 Subject: [py-dev] coverage (was issue101) In-Reply-To: <20100526125701.GQ17693@trillke.net> References: <1b120fc54ab73ca097eae5314e4d9af3@bitbucket.org> <20100525103218.GL17693@trillke.net> <20100526125701.GQ17693@trillke.net> Message-ID: Hi, > i actually now added your pytest-cov plugin to the plugin index page: > > http://pytest.org/plugin/index.html > > :) We now advertise two coverage plugins, not a too fortunate situation. Yay, thx! > seems like your plugin now works much better for me than some versions ago. > I still find your command line option design a bit obnoxious - i usually > try to keep it much more minimal. Are you open to advise/suggestions there? With the early release I noticed that it got out of sync a couple of times since it depended on specific dev versions and I was chasing, but I think that has stabilised more now. Yes open to discussion. I have already thought about the options issue and played with some alternatives but not yet good replacement happy with, but definitely on the todo. Initially I wanted to support all of coverage since part of reason to write was that I couldn't do what I needed. I'm sure the options can be refactored down to a better set while still giving good control. > Also, is it possible to start a Python script/subprocess and get its coverage > included as well? pylib/.test itself has a lot of functional tests and > those start subprocesses which is a usual scenario. This is a good idea and I think that it is possible, I will have to look into the details. :) From issues-noreply at bitbucket.org Sat May 29 15:47:47 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Sat, 29 May 2010 13:47:47 -0000 Subject: [py-dev] New issue 103 in py-trunk: Introduce "with py.test.raises(...)" Message-ID: <08e74fa16004514f2525d9e0d595ae39@bitbucket.org> --- you can reply above this line --- New issue 103: Introduce "with py.test.raises(...)" http://bitbucket.org/hpk42/py-trunk/issue/103/introduce-with-pytestraises holger krekel / hpk42 on Sat, 29 May 2010 15:47:47 +0200: Description: For those who want to keep their code compatible to >=Python2.6 it'd be nice to extend py.test.raises with a third form: {{{ with py.test.raises(ValueError) as excinfo: #codeblock }}} which would be equivalent to using the previous string-notation: {{{ excinfo = py.test.raises(ValueError, "#codeblock") }}} I believe this could be implemented in a manner that preserves py.test's single-source nature for Python2.4-Python3.1. And i wonder if there are other potential extensions/uses for newer syntax. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.