From nicoe at no-log.org Sun Apr 3 23:51:26 2011 From: nicoe at no-log.org (Nicolas =?utf-8?Q?=C3=89vrard?=) Date: Sun, 3 Apr 2011 23:51:26 +0200 Subject: [py-dev] How can I use py.test in trytond Message-ID: <20110403215126.GA28374@nutellux.dyndns.org> Hello, I develop on tryton[1] and I'd like to use py.test to test the new modules I am writing for it. This framework has a concept of modularity that allows some module to override the definition of objects. An example will probably help understanding the situation. So we have the base trytond module which contains a `modules` directory with the list of available modules (they are all imported but not necessarily installed). project/ bin/ trytond/ modules/ ... sale/ sale_cost/ ... So when you install the sale_cost module the object defined on the sale module are extended with more fields and the methods might be extended to fit the business case. Currently I use a conftest.py file in the top directory in order to define funcargs for my tests. But it looks like a suboptimal solution. I think I can use plugins in order to do the following: 1. Define a funcarg `sale` in the sale module 2. Define a funcarg `sale` in the sale_cost module. This funcarg should ideally use the funcarg from `sale` and extend it with various information (if allowed to do so by the business rule). 3. Calling py.test trytond/modules/sale_cost from the project/ directory should work This setup would allow me to provide funcargs for use to other developers on which they can base their own tests. Right now I noticed that the funcargs are loaded recursively. Do you think it is possible to write a plugin that would read the dependency information in the sale_cost module and use it to setup the conftests file found in those modules to be imported *before* the conftest.py from sale_cost ? If it is, do you have any pointers on how I can do this ? BTW, py.test is a really nice piece of software and I really enjoy using it. [1]: http://www.tryton.org/ -- (?> Nicolas ?vrard ( ) Li?ge `? From holger at merlinux.eu Mon Apr 4 09:37:36 2011 From: holger at merlinux.eu (holger krekel) Date: Mon, 4 Apr 2011 07:37:36 +0000 Subject: [py-dev] How can I use py.test in trytond In-Reply-To: <20110403215126.GA28374@nutellux.dyndns.org> References: <20110403215126.GA28374@nutellux.dyndns.org> Message-ID: <20110404073736.GS16231@merlinux.eu> Hi Nicolas, On Sun, Apr 03, 2011 at 23:51 +0200, Nicolas ?vrard wrote: > Hello, > > I develop on tryton[1] and I'd like to use py.test to test the new > modules I am writing for it. > > This framework has a concept of modularity that allows some module to > override the definition of objects. An example will probably help > understanding the situation. > > So we have the base trytond module which contains a `modules` > directory with the list of available modules (they are all imported > but not necessarily installed). > > project/ > bin/ > trytond/ > modules/ > ... > sale/ > sale_cost/ > ... > > So when you install the sale_cost module the object defined on the > sale module are extended with more fields and the methods might be > extended to fit the business case. > > Currently I use a conftest.py file in the top directory in order to > define funcargs for my tests. But it looks like a suboptimal solution. > > I think I can use plugins in order to do the following: > > 1. Define a funcarg `sale` in the sale module > 2. Define a funcarg `sale` in the sale_cost module. This funcarg > should ideally use the funcarg from `sale` and extend it with > various information (if allowed to do so by the business rule). > 3. Calling py.test trytond/modules/sale_cost from the project/ > directory should work > > This setup would allow me to provide funcargs for use to other > developers on which they can base their own tests. > > Right now I noticed that the funcargs are loaded recursively. Do you > think it is possible to write a plugin that would read the dependency > information in the sale_cost module and use it to setup the conftests > file found in those modules to be imported *before* the conftest.py > from sale_cost ? > > If it is, do you have any pointers on how I can do this ? request.getfuncargvalue() can be called to decorate an existing funcarg, see http://bit.ly/eiByLF However, i guess you could just have a single conftest.py which provides different values for your funcargs depending on which directory they are requested from. (``request.module.fspath`` gives you the file system path of the python module.) This way you would encode all the logic in one place. HTH, holger > BTW, py.test is a really nice piece of software and I really enjoy > using it. BTW, nice to hear :) > > [1]: http://www.tryton.org/ > > -- > (?> Nicolas ?vrard > ( ) Li?ge > `? > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev From nicoe at no-log.org Mon Apr 4 13:03:24 2011 From: nicoe at no-log.org (Nicolas =?utf-8?Q?=C3=89vrard?=) Date: Mon, 4 Apr 2011 13:03:24 +0200 Subject: [py-dev] How can I use py.test in trytond In-Reply-To: <20110404073736.GS16231@merlinux.eu> References: <20110403215126.GA28374@nutellux.dyndns.org> <20110404073736.GS16231@merlinux.eu> Message-ID: <20110404110324.GA24950@nutellux.dyndns.org> * holger krekel [2011-04-04 09:37 +0200]: >Hi Nicolas, Hello, >> Currently I use a conftest.py file in the top directory in order to >> define funcargs for my tests. But it looks like a suboptimal solution. >> >> I think I can use plugins in order to do the following: >> >> 1. Define a funcarg `sale` in the sale module >> 2. Define a funcarg `sale` in the sale_cost module. This funcarg >> should ideally use the funcarg from `sale` and extend it with >> various information (if allowed to do so by the business rule). >> 3. Calling py.test trytond/modules/sale_cost from the project/ >> directory should work >> >> This setup would allow me to provide funcargs for use to other >> developers on which they can base their own tests. >> >> Right now I noticed that the funcargs are loaded recursively. Do you >> think it is possible to write a plugin that would read the dependency >> information in the sale_cost module and use it to setup the conftests >> file found in those modules to be imported *before* the conftest.py >> from sale_cost ? >> >> If it is, do you have any pointers on how I can do this ? > >request.getfuncargvalue() can be called to decorate an existing funcarg, >see http://bit.ly/eiByLF Thank you for the link I missed it while going through the doc (although I knew that you could call getfuncargvalue to get the 'previous' value and play with it). >However, i guess you could just have a single conftest.py which provides different >values for your funcargs depending on which directory they are requested from. That's precisely what I don't want to do. Because modules are distributed through pypi and thus you never know which modules are installed. Moreover, I think that such a setup would have many difficulties to address all the possible cases (modules incompatibilities, bad dependency specified in the __tryton__ file, and so on). My idea is the following: - During test conftest collection, locate the nearest __tryton__.py file (by going in the direction of the root of the file system). - Once you found it, collect also conftest information from the modules specified in this file (the modules are usually in a sibling directory from the directory where you found the __tryton__.py file but they may also be found thanks to their entry point). - The collection should of course work recursively It is important not to load all conftest information (because one might have conflicting configuration for the same object) but only the one you depend on. Is this kind of setup doable in a plugin ? Am I completely wrong about how this problematic should be addressed ? >> BTW, py.test is a really nice piece of software and I really enjoy >> using it. > >BTW, nice to hear :) It is well deserved :). -- (?> Nicolas ?vrard ( ) Li?ge `? From prologic at shortcircuit.net.au Tue Apr 5 01:00:58 2011 From: prologic at shortcircuit.net.au (James Mills) Date: Tue, 5 Apr 2011 09:00:58 +1000 Subject: [py-dev] py.test + CI Message-ID: Hey all, Currently what options are available for Continuous Integration with py.test (or any other test suite / runner for that matter) ? cheers James -- -- James Mills -- -- "Problems are solved by method" From flub at devork.be Tue Apr 5 14:24:36 2011 From: flub at devork.be (Floris Bruynooghe) Date: Tue, 5 Apr 2011 13:24:36 +0100 Subject: [py-dev] py.test + CI In-Reply-To: References: Message-ID: On 5 April 2011 00:00, James Mills wrote: > Currently what options are available for > Continuous Integration with py.test (or any other test suite / runner > for that matter) ? py.test has a --junitxml option which writes reports in the de-facto JUnit XML format. E.g. Jenkins/Hudson can read this XML to do detailed reporting. I assume there's more tools that understand that format though never researched this. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From rafalskiy at gmail.com Tue Apr 5 17:42:15 2011 From: rafalskiy at gmail.com (Vyacheslav Rafalskiy) Date: Tue, 5 Apr 2011 11:42:15 -0400 Subject: [py-dev] migration from 1.3 to 2.0 Message-ID: Hi Holger and list members, I have been using py.test for some time after switching from nose and quite happy with it. However, when migrating from 1.3.1 to 2.0.2 I hit a couple of snags, which makes me hesitate. Perhaps you can help me. 1. Just to clear the point. I understand from another post that option_xxxx variables in conftest.py are gone and recommended replacement is to use pytest_cmdline_preparse(). Is that so? 2. The new behavior is that even if you run py.test --help, the pytest_configure() is still run. In my case pytest_configure() does quite a bit of heavy lifting and raises exceptions at wrong parameters. The result is that if the parameter is wrong I cannot use help to find out what is the right parameter. Of course, the workaround could be: def pytest_configure(config): if config.getvalue('help'): return ... but it seems ugly. Am I in a wrong hook? 3. In my pytest_configure() there are numerous conditions when it can fail. For this conditions I have exceptions with specially crafted messages, intended for different people. Now they are gone, replaced by a long and scary listing with every line prepended with INTERNALERROR. What is internal about it? Can I continue managing my configuration errors? Thanks, Vyacheslav From holger at merlinux.eu Tue Apr 5 22:16:12 2011 From: holger at merlinux.eu (holger krekel) Date: Tue, 5 Apr 2011 20:16:12 +0000 Subject: [py-dev] migration from 1.3 to 2.0 In-Reply-To: References: Message-ID: <20110405201612.GW16231@merlinux.eu> Hi Vyachselav, On Tue, Apr 05, 2011 at 11:42 -0400, Vyacheslav Rafalskiy wrote: > Hi Holger and list members, > > I have been using py.test for some time after switching from nose and > quite happy with it. > However, when migrating from 1.3.1 to 2.0.2 I hit a couple of snags, > which makes me hesitate. > Perhaps you can help me. can try but am only online intermittently currently. > 1. Just to clear the point. I understand from another post that > option_xxxx variables > in conftest.py are gone and recommended replacement is to use > pytest_cmdline_preparse(). > Is that so? Not exactly. You can use a .ini file to add command line options: http://pytest.org/customize.html#adding-default-options Or do you need to do something more dynamic? In that case you could play with pytest_cmdline_preparse, indeed. > 2. The new behavior is that even if you run py.test --help, the > pytest_configure() is still run. > In my case pytest_configure() does quite a bit of heavy lifting and > raises exceptions at > wrong parameters. The result is that if the parameter is wrong I > cannot use help to find > out what is the right parameter. Of course, the workaround could be: > > def pytest_configure(config): > if config.getvalue('help'): > return > ... > > but it seems ugly. Am I in a wrong hook? You could try pytest_sessionstart(session) and access session.config for the config object. > 3. In my pytest_configure() there are numerous conditions when it can > fail. For this > conditions I have exceptions with specially crafted messages, intended > for different > people. Now they are gone, replaced by a long and scary listing with every line > prepended with INTERNALERROR. What is internal about it? Can I continue managing > my configuration errors? Sure, you should be able to. I guess it was a bit of an incident that failures in pytest_configure were shown nicely. I am not quite sure immediately what the best way to relay "global" configuration messages to the user is. If you'd use "funcargs" and the "session" scope for setting up resources then it would show nicely. Feel free to open a feature/enhancement request to have py.test show failures in sessionstart or configure hooks in a way that helps you. This way we can write a specific test and make sure it works also in the future. best, holger From prologic at shortcircuit.net.au Wed Apr 6 01:16:54 2011 From: prologic at shortcircuit.net.au (James Mills) Date: Wed, 6 Apr 2011 09:16:54 +1000 Subject: [py-dev] py.test + CI In-Reply-To: References: Message-ID: On Tue, Apr 5, 2011 at 10:24 PM, Floris Bruynooghe wrote: > py.test has a --junitxml option which writes reports in the de-facto > JUnit XML format. ?E.g. Jenkins/Hudson can read this XML to do > detailed reporting. ?I assume there's more tools that understand that > format though never researched this. Yeah I've seen this looking through what py.test can output... I don't know about anyone else though... But I kind of feel like we need a new set of CI tools for Python. Hudson would not work out of the box for me and I get frustrated easily be complexity and things that don't work as "advertised" :) (Even buildbot didn't work) cheers James -- -- James Mills -- -- "Problems are solved by method" From Ronny.Pfannschmidt at gmx.de Wed Apr 6 01:51:55 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Wed, 06 Apr 2011 01:51:55 +0200 Subject: [py-dev] py.test + CI In-Reply-To: References: Message-ID: <1302047515.10381.32.camel@Klappe2> On Wed, 2011-04-06 at 09:16 +1000, James Mills wrote: > On Tue, Apr 5, 2011 at 10:24 PM, Floris Bruynooghe wrote: > > py.test has a --junitxml option which writes reports in the de-facto > > JUnit XML format. E.g. Jenkins/Hudson can read this XML to do > > detailed reporting. I assume there's more tools that understand that > > format though never researched this. > > Yeah I've seen this looking through what py.test can output... > > I don't know about anyone else though... But I kind of feel like > we need a new set of CI tools for Python. Hudson would not work > out of the box for me and I get frustrated easily be complexity > and things that don't work as "advertised" :) (Even buildbot didn't work) there are various people that feel the need for more pythonic CI tools, but currently seems like the time is not right to just gather them for hacking it up its on my wishlist as well, but until anyvc is ready as base lib i wont take a look at the problem again (last time i failed bad) -- Ronny -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From holger at merlinux.eu Wed Apr 6 09:01:36 2011 From: holger at merlinux.eu (holger krekel) Date: Wed, 6 Apr 2011 07:01:36 +0000 Subject: [py-dev] py.test + CI In-Reply-To: References: Message-ID: <20110406070136.GY16231@merlinux.eu> On Wed, Apr 06, 2011 at 09:16 +1000, James Mills wrote: > On Tue, Apr 5, 2011 at 10:24 PM, Floris Bruynooghe wrote: > > py.test has a --junitxml option which writes reports in the de-facto > > JUnit XML format. ?E.g. Jenkins/Hudson can read this XML to do > > detailed reporting. ?I assume there's more tools that understand that > > format though never researched this. > > Yeah I've seen this looking through what py.test can output... > > I don't know about anyone else though... But I kind of feel like > we need a new set of CI tools for Python. Hudson would not work > out of the box for me and I get frustrated easily be complexity > and things that don't work as "advertised" :) (Even buildbot didn't work) I much understand the sentiment and hope to work myself towards a better solution some day :) Meanwhile you might want to look into tox (http://codespeak.net/tox) which helps to automate environment setup. Given you have a "tox.ini" configuration working for your project you can then relatively easily integrate it with Jenkins/Hudson. holger > cheers > James > > -- > -- James Mills > -- > -- "Problems are solved by method" > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev From holger at merlinux.eu Wed Apr 6 10:04:49 2011 From: holger at merlinux.eu (holger krekel) Date: Wed, 6 Apr 2011 08:04:49 +0000 Subject: [py-dev] How can I use py.test in trytond In-Reply-To: <20110404110324.GA24950@nutellux.dyndns.org> References: <20110403215126.GA28374@nutellux.dyndns.org> <20110404073736.GS16231@merlinux.eu> <20110404110324.GA24950@nutellux.dyndns.org> Message-ID: <20110406080449.GZ16231@merlinux.eu> On Mon, Apr 04, 2011 at 13:03 +0200, Nicolas ?vrard wrote: > * holger krekel [2011-04-04 09:37 +0200]: > >Hi Nicolas, > > Hello, > > >>Currently I use a conftest.py file in the top directory in order to > >>define funcargs for my tests. But it looks like a suboptimal solution. > >> > >>I think I can use plugins in order to do the following: > >> > >> 1. Define a funcarg `sale` in the sale module > >> 2. Define a funcarg `sale` in the sale_cost module. This funcarg > >> should ideally use the funcarg from `sale` and extend it with > >> various information (if allowed to do so by the business rule). > >> 3. Calling py.test trytond/modules/sale_cost from the project/ > >> directory should work > >> > >>This setup would allow me to provide funcargs for use to other > >>developers on which they can base their own tests. > >> > >>Right now I noticed that the funcargs are loaded recursively. Do you > >>think it is possible to write a plugin that would read the dependency > >>information in the sale_cost module and use it to setup the conftests > >>file found in those modules to be imported *before* the conftest.py > >>from sale_cost ? > >> > >>If it is, do you have any pointers on how I can do this ? > > > >request.getfuncargvalue() can be called to decorate an existing funcarg, > >see http://bit.ly/eiByLF > > Thank you for the link I missed it while going through the doc > (although I knew that you could call getfuncargvalue to get the > 'previous' value and play with it). > > >However, i guess you could just have a single conftest.py which provides different > >values for your funcargs depending on which directory they are requested from. > > That's precisely what I don't want to do. OK, i didn't read your first post correctly. > Because modules are distributed through pypi and thus you never know > which modules are installed. Moreover, I think that such a setup would > have many difficulties to address all the possible cases (modules > incompatibilities, bad dependency specified in the __tryton__ file, > and so on). > > My idea is the following: > > - During test conftest collection, locate the nearest > __tryton__.py file (by going in the direction of the root of the > file system). > - Once you found it, collect also conftest information from > the modules specified in this file (the modules are usually in a > sibling directory from the directory where you found the > __tryton__.py file but they may also be found thanks to their > entry point). > - The collection should of course work recursively > > It is important not to load all conftest information (because one > might have conflicting configuration for the same object) but only the > one you depend on. > > Is this kind of setup doable in a plugin ? Hum, might be tricky. To begin with, conftest.py detection/looading is mostly done via internal APIs. Second, conftest.py definitions are only consulted for the particularly subdirectory so that for a/conftest.py b/ conftest.py test_whatever.py for the path "b/test_whatever.py" funcargs defined in a/conftest.py file will not be consulted. Maybe it's best if you experiment with your own convention. An idea could be to define a pytest_cmdline_main() hook in your top-level conftest.py file (which you can morph to become a pypi-installable installable pytest-tryton plugin once it works nicely). In this hook you would look for your __tryton__.py file, load it and look for which modules need to be loaded. In the module dirs you could have a "conftestresource.py" module which defines funcargs as if you would do it in a conftest.py file. You could import this module and register it as a plugin to py.test like this:: config.pluginmanager.register(pluginmodule_or_object, name, prepend=True) # or False This way your modules grow the ability to interact with testing and you could even use "request.getfuncargvalue()" given that your project definitions and thus the registrations of funcarg resources are accordingly ordered. If not you might be able to use the "@pytest.mark.tryfirst" marker on any base object which modules want to enhance. You might want to read the _pytest/*.py source code a bit, grepping for the bits and pieces i posted. > Am I completely wrong about how this problematic should be addressed ? Depends on if i am wrong about what i am suggesting for your situation :) best, holger > >>BTW, py.test is a really nice piece of software and I really enjoy > >>using it. > > > >BTW, nice to hear :) > > It is well deserved :). > > -- > (?> Nicolas ?vrard > ( ) Li?ge > `? > From holger at merlinux.eu Wed Apr 6 13:48:05 2011 From: holger at merlinux.eu (holger krekel) Date: Wed, 6 Apr 2011 11:48:05 +0000 Subject: [py-dev] latest pytest questions for pytest-cov In-Reply-To: References: <20110320112120.GL16231@merlinux.eu> Message-ID: <20110406114805.GB16231@merlinux.eu> Hey Meme, On Sun, Mar 27, 2011 at 22:28 +1100, meme dough wrote: > Hi, > > Now up and working with the latest pytest (think only distributed > testing was an issue). It is still backward compatible with older > versions of pytest (before the split). > > Funcarg is at module level so visible with py.test --funcargs. > > pth file more robust since it doesn't complain if cov-core got deleted > and pth file left. > > More doc for coverage config file since couple of people have asked questions. > > I moved the doc to the docstring in the module. I see that the > website just points to the external plugins at pypi, so that means the > doc can't get out of sync right? right, i'd think so :) nice changes. holger > :) > > On 20 March 2011 22:21, holger krekel wrote: > > Hi Meme, > > > > On Sun, Mar 20, 2011 at 22:00 +1100, meme dough wrote: > >> Hi, > >> > >> Currently looking at fixing pytest-cov to work with latest pytest. ?I > >> see two issues so far. > >> > >> 1. How to determine if non distributed, or distributed master, or > >> distributed slave. > >> > >> This used to look at the session class Session = Central (non > >> distributed), DSession = distributed master, and SlaveSession = > >> distributed slave. > >> > >> So now I'm thinking look at > >> session.config.pluginmanager.hasplugin('dsession') which means > >> distributed master. ?For distributed slave check if config.slaveinput > >> is there which means distributed slave. ?If none of those then must be > >> non distributed. > > > > I guess that's ok. ?There is no "official" way at the moment. > > > >> Is there a better way to determine at pytest_sessionstart if the > >> process is central (non distributed), distributed master or > >> distributed slave? > >> > >> 2. Need to know topdir. > >> > >> This used to use config.topdir to know what the root dir is. ?This is > >> needed since distributed testing may be co-located or not. ?If not > >> then have to rewrite all source paths back to what the master is set > >> to so that all coverage files combine correctly even if have many > >> separate tx envs on diff machines on diff paths. > >> > >> Has this moved somewhere? ?I see config.fspath, is that it? ?I look > >> more I just got to this bit. > > > > No, there now is a "looponfailroots" (see --help) and distributed > > testing uses "rsyncroots". The problem is that "topdir" was a bit > > of an artifical concept. > > > > Maybe it helps, that item.nodeid contains a "::" separated test id where > > the first part is a path relative to the current directory and > > normalized to use "/" as separator. > > > > best, > > holger > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From rafalskiy at gmail.com Wed Apr 6 18:37:19 2011 From: rafalskiy at gmail.com (Vyacheslav Rafalskiy) Date: Wed, 6 Apr 2011 12:37:19 -0400 Subject: [py-dev] migration from 1.3 to 2.0 In-Reply-To: <20110405201612.GW16231@merlinux.eu> References: <20110405201612.GW16231@merlinux.eu> Message-ID: Thanks Holger, On Tue, Apr 5, 2011 at 4:16 PM, holger krekel wrote: > Hi Vyachselav, > > On Tue, Apr 05, 2011 at 11:42 -0400, Vyacheslav Rafalskiy wrote: >> Hi Holger and list members, >> >> I have been using py.test for some time after switching from nose and >> quite happy with it. >> However, when migrating from 1.3.1 to 2.0.2 I hit a couple of snags, >> which makes me hesitate. >> Perhaps you can help me. > > can try but am only online intermittently currently. > >> 1. Just to clear the point. I understand from another post that >> option_xxxx variables >> in conftest.py are gone and recommended replacement is to use >> pytest_cmdline_preparse(). >> Is that so? > > Not exactly. ?You can use a .ini file to add command line options: > > ? ?http://pytest.org/customize.html#adding-default-options I wouldn't want to maintain an extra config file. I already have conftest.py with a lot of hardcoded stuff, a bit more does not hurt. Moreover, if you put it like def pytest_cmdline_preparse(args): args[0:0] = [ '--verbose', '--capture=no', ] it looks even cleaner than using option_xxxx variables in spite of a bit of boilerplate. >> 3. In my pytest_configure() there are numerous conditions when it can >> fail. For this >> conditions I have exceptions with specially crafted messages, intended >> for different >> people. Now they are gone, replaced by a long and scary listing with every line >> prepended with INTERNALERROR. What is internal about it? Can I continue managing >> my configuration errors? > > Sure, you should be able to. I guess it was a bit of an incident that > failures in pytest_configure were shown nicely. ?I am not quite sure > immediately what the best way to relay "global" configuration messages > to the user is. ?If you'd use "funcargs" and the "session" scope for > setting up resources then it would show nicely. ?Feel free to open > a feature/enhancement request to have py.test show failures > in sessionstart or configure hooks in a way that helps you. > This way we can write a specific test and make sure it works > also in the future. If I understand correctly, using funcargs means that every test function of hundreds I have in several suites should include a reference to the global setup funcarg. It seems a non-starter. I guess I will stick to the old version and try to think of something to enter in a feature request. Vyacheslav From rafalskiy at gmail.com Wed Apr 6 23:00:59 2011 From: rafalskiy at gmail.com (Vyacheslav Rafalskiy) Date: Wed, 6 Apr 2011 17:00:59 -0400 Subject: [py-dev] migration from 1.3 to 2.0 In-Reply-To: References: <20110405201612.GW16231@merlinux.eu> Message-ID: >>> 3. In my pytest_configure() there are numerous conditions when it can >>> fail. For this >>> conditions I have exceptions with specially crafted messages, intended >>> for different >>> people. Now they are gone, replaced by a long and scary listing with every line >>> prepended with INTERNALERROR. What is internal about it? Can I continue managing >>> my configuration errors? >> >> Sure, you should be able to. I guess it was a bit of an incident that >> failures in pytest_configure were shown nicely. ?I am not quite sure >> immediately what the best way to relay "global" configuration messages >> to the user is. ?If you'd use "funcargs" and the "session" scope for >> setting up resources then it would show nicely. ?Feel free to open >> a feature/enhancement request to have py.test show failures >> in sessionstart or configure hooks in a way that helps you. >> This way we can write a specific test and make sure it works >> also in the future. > > If I understand correctly, using funcargs means that every test function > of hundreds I have in several suites should include a reference to the > global setup funcarg. It seems a non-starter. > > I guess I will stick to the old version and try to think of something > ?to enter in a feature request. > > Vyacheslav > Looks like I solved my main problem with a monkey patch, admittedly a brutal one import sys import _pytest.core def notify_exception(self, excinfo): excrepr = excinfo.getrepr(style='native') sys.stderr.write(excrepr) _pytest.core.PluginManager.notify_exception = notify_exception From holger at merlinux.eu Thu Apr 7 08:17:15 2011 From: holger at merlinux.eu (holger krekel) Date: Thu, 7 Apr 2011 06:17:15 +0000 Subject: [py-dev] migration from 1.3 to 2.0 In-Reply-To: References: <20110405201612.GW16231@merlinux.eu> Message-ID: <20110407061715.GE16231@merlinux.eu> On Wed, Apr 06, 2011 at 12:37 -0400, Vyacheslav Rafalskiy wrote: > > Not exactly. ?You can use a .ini file to add command line options: > > > > ? ?http://pytest.org/customize.html#adding-default-options > > I wouldn't want to maintain an extra config file. I already have > conftest.py with > a lot of hardcoded stuff, a bit more does not hurt. Moreover, if you put it like > > def pytest_cmdline_preparse(args): > args[0:0] = [ > '--verbose', > '--capture=no', > ] > > it looks even cleaner than using option_xxxx variables in spite of a > bit of boilerplate. Sure, if you have a root conftest.py anyway that works. Note that you can put the .ini config into a setup.cfg as well which is used by the upcoming python package distribution as well (and setuptools). > >> 3. In my pytest_configure() there are numerous conditions when it can > >> fail. For this > >> conditions I have exceptions with specially crafted messages, intended > >> for different > >> people. Now they are gone, replaced by a long and scary listing with every line > >> prepended with INTERNALERROR. What is internal about it? Can I continue managing > >> my configuration errors? > > > > Sure, you should be able to. I guess it was a bit of an incident that > > failures in pytest_configure were shown nicely. ?I am not quite sure > > immediately what the best way to relay "global" configuration messages > > to the user is. ?If you'd use "funcargs" and the "session" scope for > > setting up resources then it would show nicely. ?Feel free to open > > a feature/enhancement request to have py.test show failures > > in sessionstart or configure hooks in a way that helps you. > > This way we can write a specific test and make sure it works > > also in the future. > > If I understand correctly, using funcargs means that every test function > of hundreds I have in several suites should include a reference to the > global setup funcarg. It seems a non-starter. > > I guess I will stick to the old version and try to think of something > to enter in a feature request. sure, makes sense. isn't your main issue that upon actual test start (after collection) you want to do some configuration steps which might result in error conditions which you'd like to see nicely reported to the user? (sidenote: configure and even sessionstart hooks are both a bit not 100% right because they happen even on the master side of a distributed test run and the master side does not collect or run tests at all) best, holger > Vyacheslav > From rafalskiy at gmail.com Thu Apr 7 18:29:09 2011 From: rafalskiy at gmail.com (Vyacheslav Rafalskiy) Date: Thu, 7 Apr 2011 12:29:09 -0400 Subject: [py-dev] migration from 1.3 to 2.0 In-Reply-To: <20110407061715.GE16231@merlinux.eu> References: <20110405201612.GW16231@merlinux.eu> <20110407061715.GE16231@merlinux.eu> Message-ID: >> >> 3. In my pytest_configure() there are numerous conditions when it can >> >> fail. For this >> >> conditions I have exceptions with specially crafted messages, intended >> >> for different >> >> people. Now they are gone, replaced by a long and scary listing with every line >> >> prepended with INTERNALERROR. What is internal about it? Can I continue managing >> >> my configuration errors? >> > >> > Sure, you should be able to. I guess it was a bit of an incident that >> > failures in pytest_configure were shown nicely. ?I am not quite sure >> > immediately what the best way to relay "global" configuration messages >> > to the user is. ?If you'd use "funcargs" and the "session" scope for >> > setting up resources then it would show nicely. ?Feel free to open >> > a feature/enhancement request to have py.test show failures >> > in sessionstart or configure hooks in a way that helps you. >> > This way we can write a specific test and make sure it works >> > also in the future. >> >> If I understand correctly, using funcargs means that every test function >> of hundreds I have in several suites should include a reference to the >> global setup funcarg. It seems a non-starter. >> >> I guess I will stick to the old version and try to think of something >> ?to enter in a feature request. > > sure, makes sense. ?isn't your main issue that upon actual test start > (after collection) you want to do some configuration steps which might > result in error conditions which you'd like to see nicely reported to > the user? That's exactly my point. >(sidenote: configure and even sessionstart hooks are both a bit > not 100% right because they happen even on the master side of a distributed > test run and the master side does not collect or run tests at all) I see. Perhaps something like setup_package() in the top-level __init__.py could be a solution? Vyacheslav From flub at devork.be Tue Apr 12 00:55:40 2011 From: flub at devork.be (Floris Bruynooghe) Date: Mon, 11 Apr 2011 23:55:40 +0100 Subject: [py-dev] Creating unicode strings in pytest testsuit Message-ID: Hi While working on the patch for the invalid unicode codepoints in the junitxml output I discovered it's fairly hard to create a unicode strings in the the test suite. Currently I decided to use py.builtins._totext() to replace unicode string literals. And to get access to the unichr() builtin I've done this little hack: def foo(): global unichr try: unichr(65) except NameError: unichr = chr I think both are fairly fine, though am surprised by the leading underscore in _totext(), I'd have rather expected a py.builtin.unicode() name or so. And if the unichr hack is acceptable would it be worth to create py.builtins.unichr()? Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From Ronny.Pfannschmidt at gmx.de Wed Apr 13 00:38:34 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Wed, 13 Apr 2011 00:38:34 +0200 Subject: [py-dev] the concept of a per test "test dir" and its uses Message-ID: <1302647914.5537.439.camel@Klappe2> hi, after some thought i would like to discuss adding a generic testdir, so for each test different plugins could store data/metadata currently only the tempdir plugin creates a session dir on first use of the tmpdir funcarg/pytest.mktemp i'd rather have to possibility to store for each test * full test id * stdout/err * logfile for stdlib logging * tmpdir for plugin * some kind of serialized full taceback, optionally with some locals/globals * coverage reports * profiler reports since all of those are a misfit for the tmpdir functionality, i'd like to propose a extension of the mechanism the basic tmpdir would move to the "tmpdir" subdir of the testdir, stdout/err, log, would in turn be files in the testdir it would also be easily possible to keep those in sync with xdist+rsync should the need arise, just make a new testdir for the remote id, and sync the directory content. further possible uses include using coverage data and test results of previous runs to decide what tests can likely be skipped without causing too much of a problem or providing relative progress information on the collection status (based on the number of test id's in the last run). Please discuss additional uses and possible problems. Regards Ronny -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From holger at merlinux.eu Wed Apr 13 13:53:14 2011 From: holger at merlinux.eu (holger krekel) Date: Wed, 13 Apr 2011 11:53:14 +0000 Subject: [py-dev] Creating unicode strings in pytest testsuit In-Reply-To: References: Message-ID: <20110413115314.GO16231@merlinux.eu> Hey Floris, On Mon, Apr 11, 2011 at 23:55 +0100, Floris Bruynooghe wrote: > Hi > > While working on the patch for the invalid unicode codepoints in the > junitxml output I discovered it's fairly hard to create a unicode > strings in the the test suite. Currently I decided to use > py.builtins._totext() to replace unicode string literals. And to get > access to the unichr() builtin I've done this little hack: > > def foo(): > global unichr > try: > unichr(65) > except NameError: > unichr = chr > > I think both are fairly fine, though am surprised by the leading > underscore in _totext(), I'd have rather expected a > py.builtin.unicode() name or so. And if the unichr hack is acceptable > would it be worth to create py.builtins.unichr()? i guess this makes sense so introducing a unicode and unicode (as a duplicate for _totext for now) would make sense. Btw, i'd like to do a py/pytest release around the coming weekend - would be cool to have your fix in, even if the test isn't written in the optimal way yet. best & thanks, holger > Regards > Floris > > -- > Debian GNU/Linux -- The Power of Freedom > www.debian.org | www.gnu.org | www.kernel.org > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From holger at merlinux.eu Wed Apr 13 14:02:52 2011 From: holger at merlinux.eu (holger krekel) Date: Wed, 13 Apr 2011 12:02:52 +0000 Subject: [py-dev] the concept of a per test "test dir" and its uses In-Reply-To: <1302647914.5537.439.camel@Klappe2> References: <1302647914.5537.439.camel@Klappe2> Message-ID: <20110413120252.GP16231@merlinux.eu> Hey Ronny, On Wed, Apr 13, 2011 at 00:38 +0200, Ronny Pfannschmidt wrote: > hi, > > after some thought i would like to discuss adding a generic testdir, > so for each test different plugins could store data/metadata > > currently only the tempdir plugin creates a session dir on first use of > the tmpdir funcarg/pytest.mktemp > > i'd rather have to possibility to store for each test > * full test id > * stdout/err > * logfile for stdlib logging > * tmpdir for plugin > * some kind of serialized full taceback, > optionally with some locals/globals > * coverage reports > * profiler reports > > since all of those are a misfit for the tmpdir functionality, > i'd like to propose a extension of the mechanism > > the basic tmpdir would move to the "tmpdir" subdir of the testdir, > stdout/err, log, would in turn be files in the testdir > > it would also be easily possible to keep those in sync with xdist+rsync > should the need arise, just make a new testdir for the remote id, and > sync the directory content. > > further possible uses include using coverage data and test results of > previous runs to decide what tests can likely be skipped without causing > too much of a problem or providing relative progress information on the > collection status (based on the number of test id's in the last run). > > Please discuss additional uses and possible problems. Currently, i'd think going for a "testdir" that easily allows to create dirs, files, run (sub) processes and collect results is one thing. And the other thing, collecting and accessing all kinds of historic information from a test run does not need to be accessible via a funcarg, does it? IMO, a pytest-history plugin would introduce recording all this information and adding cmdline opts for re-run/re-present info to access it as well as an API (pytest.gethistory() or so) best, holger > Regards Ronny > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev From schmir at gmail.com Wed Apr 13 16:15:26 2011 From: schmir at gmail.com (schmir at gmail.com) Date: Wed, 13 Apr 2011 16:15:26 +0200 Subject: [py-dev] KeyError in xdist/looponfail.py Message-ID: <87vcyinsox.fsf@muni.brainbot.com> py.test --looponfail did fail with the following traceback: ####################################### waiting for changes ######################################## ### Watching: /home/ralf/bbot/py-nextbot/nextbot/core Traceback (most recent call last): File "/home/ralf/local/bin/py.test", line 9, in load_entry_point('pytest==2.0.2', 'console_scripts', 'py.test')() File "/home/ralf/local/lib/python2.7/site-packages/_pytest/core.py", line 448, in main exitstatus = hook.pytest_cmdline_main(config=config) File "/home/ralf/local/lib/python2.7/site-packages/_pytest/core.py", line 402, in __call__ return self._docall(methods, kwargs) File "/home/ralf/local/lib/python2.7/site-packages/_pytest/core.py", line 413, in _docall res = mc.execute() File "/home/ralf/local/lib/python2.7/site-packages/_pytest/core.py", line 331, in execute res = method(**kwargs) File "/home/ralf/local/lib/python2.7/site-packages/xdist/plugin.py", line 56, in pytest_cmdline_main looponfail_main(config) File "/home/ralf/local/lib/python2.7/site-packages/xdist/looponfail.py", line 26, in looponfail_main statrecorder.waitonchange(checkinterval=2.0) File "/home/ralf/local/lib/python2.7/site-packages/xdist/looponfail.py", line 195, in waitonchange changed = self.check() File "/home/ralf/local/lib/python2.7/site-packages/xdist/looponfail.py", line 213, in check del statcache[path] KeyError: local('/home/ralf/bbot/py-nextbot/nextbot/core/searchproc_flymake.py') I'm using flymake in emacs, and searchproc_flymake.py is a temporary file generated by flymake. cheers, -- ralf From holger at merlinux.eu Thu Apr 14 18:54:38 2011 From: holger at merlinux.eu (holger krekel) Date: Thu, 14 Apr 2011 16:54:38 +0000 Subject: [py-dev] KeyError in xdist/looponfail.py In-Reply-To: <87vcyinsox.fsf@muni.brainbot.com> References: <87vcyinsox.fsf@muni.brainbot.com> Message-ID: <20110414165438.GW16231@merlinux.eu> Hey Ralf, can you give me the output of "py.test --version"? And when exactly is searchproc_flymake.py created and removed? (the looponfail-detection code should be robust against file removals but maybe there is a race condition) holger On Wed, Apr 13, 2011 at 16:15 +0200, schmir at gmail.com wrote: > py.test --looponfail did fail with the following traceback: > > ####################################### waiting for changes ######################################## > ### Watching: /home/ralf/bbot/py-nextbot/nextbot/core > Traceback (most recent call last): > File "/home/ralf/local/bin/py.test", line 9, in > load_entry_point('pytest==2.0.2', 'console_scripts', 'py.test')() > File "/home/ralf/local/lib/python2.7/site-packages/_pytest/core.py", line 448, in main > exitstatus = hook.pytest_cmdline_main(config=config) > File "/home/ralf/local/lib/python2.7/site-packages/_pytest/core.py", line 402, in __call__ > return self._docall(methods, kwargs) > File "/home/ralf/local/lib/python2.7/site-packages/_pytest/core.py", line 413, in _docall > res = mc.execute() > File "/home/ralf/local/lib/python2.7/site-packages/_pytest/core.py", line 331, in execute > res = method(**kwargs) > File "/home/ralf/local/lib/python2.7/site-packages/xdist/plugin.py", line 56, in pytest_cmdline_main > looponfail_main(config) > File "/home/ralf/local/lib/python2.7/site-packages/xdist/looponfail.py", line 26, in looponfail_main > statrecorder.waitonchange(checkinterval=2.0) > File "/home/ralf/local/lib/python2.7/site-packages/xdist/looponfail.py", line 195, in waitonchange > changed = self.check() > File "/home/ralf/local/lib/python2.7/site-packages/xdist/looponfail.py", line 213, in check > del statcache[path] > KeyError: local('/home/ralf/bbot/py-nextbot/nextbot/core/searchproc_flymake.py') > > > I'm using flymake in emacs, and searchproc_flymake.py is a temporary > file generated by flymake. > > cheers, > -- ralf > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From schmir at gmail.com Thu Apr 14 20:51:08 2011 From: schmir at gmail.com (Ralf Schmitt) Date: Thu, 14 Apr 2011 20:51:08 +0200 Subject: [py-dev] KeyError in xdist/looponfail.py In-Reply-To: <20110414165438.GW16231@merlinux.eu> (holger krekel's message of "Thu, 14 Apr 2011 16:54:38 +0000") References: <87vcyinsox.fsf@muni.brainbot.com> <20110414165438.GW16231@merlinux.eu> Message-ID: <874o60zmxv.fsf@brainbot.com> holger krekel writes: > Hey Ralf, > > can you give me the output of "py.test --version"? yes, sorry. should have mentioned that, but I was in a hurry and didn't want to let the issue get lost. ,---- | This is py.test version 2.0.2, imported from /home/ralf/local/lib/python2.7/site-packages/pytest.pyc | setuptools registered plugins: | pytest-xdist-1.5 at /home/ralf/local/lib/python2.7/site-packages/xdist/plugin.pyc `---- > And when exactly is searchproc_flymake.py created and removed? after saving searchproc.py, emacs creates searchproc_flymake.py, runs pyflakes on it and then deletes it. > (the looponfail-detection code should be robust against file > removals but maybe there is a race condition) I'm pretty sure there is a race condition :) cheers, - ralf From holger at merlinux.eu Thu Apr 14 21:37:15 2011 From: holger at merlinux.eu (holger krekel) Date: Thu, 14 Apr 2011 19:37:15 +0000 Subject: [py-dev] KeyError in xdist/looponfail.py In-Reply-To: <874o60zmxv.fsf@brainbot.com> References: <87vcyinsox.fsf@muni.brainbot.com> <20110414165438.GW16231@merlinux.eu> <874o60zmxv.fsf@brainbot.com> Message-ID: <20110414193715.GY16231@merlinux.eu> On Thu, Apr 14, 2011 at 20:51 +0200, Ralf Schmitt wrote: > holger krekel writes: > > > Hey Ralf, > > > > can you give me the output of "py.test --version"? > > yes, sorry. should have mentioned that, but I was in a hurry and didn't > want to let the issue get lost. > > ,---- > | This is py.test version 2.0.2, imported from /home/ralf/local/lib/python2.7/site-packages/pytest.pyc > | setuptools registered plugins: > | pytest-xdist-1.5 at /home/ralf/local/lib/python2.7/site-packages/xdist/plugin.pyc > `---- > > > And when exactly is searchproc_flymake.py created and removed? > > after saving searchproc.py, emacs creates searchproc_flymake.py, runs pyflakes > on it and then deletes it. > > > (the looponfail-detection code should be robust against file > > removals but maybe there is a race condition) > > I'm pretty sure there is a race condition :) i now understand which one, i think :) and fixed it in 1.6.dev4. could you try out "pip install -U -i http://pypi.testrun.org pytest-xdist"? best, holger From flub at devork.be Sat Apr 16 01:51:22 2011 From: flub at devork.be (Floris Bruynooghe) Date: Sat, 16 Apr 2011 00:51:22 +0100 Subject: [py-dev] Creating unicode strings in pytest testsuit In-Reply-To: <20110413115314.GO16231@merlinux.eu> References: <20110413115314.GO16231@merlinux.eu> Message-ID: Hello On 13 April 2011 12:53, holger krekel wrote: > Btw, i'd like to do a py/pytest release around the coming weekend - > would be cool to have your fix in, even if the test isn't written > in the optimal way yet. I've finally managed to stop confusing myself and seem to have succeeded in a more comprehensive fix. Though I did take some shortcuts in the tests as you suggested, there where some more issues with testing unicode characters so the tests just skip things outside of the ascii range. testrun.org seems happy, though two of the builds (pypy, py31) seem to be hanging on unrelated issues. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From holger at merlinux.eu Sun Apr 17 12:30:51 2011 From: holger at merlinux.eu (holger krekel) Date: Sun, 17 Apr 2011 10:30:51 +0000 Subject: [py-dev] Creating unicode strings in pytest testsuit In-Reply-To: References: <20110413115314.GO16231@merlinux.eu> Message-ID: <20110417103051.GA16231@merlinux.eu> Hi Floris, thanks, looks good. testrun.org is not too reliable these days, indeed. (I'd like to have a much simpler and more robust service than Hudson/Jenkins some day). best, holger On Sat, Apr 16, 2011 at 00:51 +0100, Floris Bruynooghe wrote: > Hello > > On 13 April 2011 12:53, holger krekel wrote: > > Btw, i'd like to do a py/pytest release around the coming weekend - > > would be cool to have your fix in, even if the test isn't written > > in the optimal way yet. > > I've finally managed to stop confusing myself and seem to have > succeeded in a more comprehensive fix. Though I did take some > shortcuts in the tests as you suggested, there where some more issues > with testing unicode characters so the tests just skip things outside > of the ascii range. > > testrun.org seems happy, though two of the builds (pypy, py31) seem to > be hanging on unrelated issues. > > Regards > Floris > > -- > Debian GNU/Linux -- The Power of Freedom > www.debian.org | www.gnu.org | www.kernel.org > From holger at merlinux.eu Sun Apr 17 23:18:38 2011 From: holger at merlinux.eu (holger krekel) Date: Sun, 17 Apr 2011 21:18:38 +0000 Subject: [py-dev] pytest-2.0.3: bug fixes and speed ups Message-ID: <20110417211838.GC16231@merlinux.eu> py.test 2.0.3: bug fixes and speed ups =========================================================================== Welcome to pytest-2.0.3, a maintenance and bug fix release of pytest, a mature testing tool for Python, supporting CPython 2.4-3.2, Jython and latest PyPy interpreters. See the extensive docs with tested examples here: http://pytest.org/ If you want to install or upgrade pytest, just type one of:: pip install -U pytest # or easy_install -U pytest There also is a bugfix release 1.6 of pytest-xdist, the plugin that enables seemless distributed and "looponfail" testing for Python. best, holger krekel Changes between 2.0.2 and 2.0.3 ---------------------------------------------- - fix issue38: nicer tracebacks on calls to hooks, particularly early configure/sessionstart ones - fix missing skip reason/meta information in junitxml files, reported via http://lists.idyll.org/pipermail/testing-in-python/2011-March/003928.html - fix issue34: avoid collection failure with "test" prefixed classes deriving from object. - don't require zlib (and other libs) for genscript plugin without --genscript actually being used. - speed up skips (by not doing a full traceback represenation internally) - fix issue37: avoid invalid characters in junitxml's output From Ronny.Pfannschmidt at gmx.de Wed Apr 20 22:47:48 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Wed, 20 Apr 2011 22:47:48 +0200 Subject: [py-dev] initial take at a generic funcags hook Message-ID: <1303332468.8027.7.camel@Klappe2> hi, i just tried a initial implementation of generic funcargs to solve the issue i introduced the hook pytest_reconfigure_funcargs(request) the current patch is at https://bitbucket.org/RonnyPfannschmidt/pytest-patches/qseries?apply=t&qs_apply=reconfigure-funcargs regards, Ronny -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From schettino72 at gmail.com Mon Apr 25 17:40:18 2011 From: schettino72 at gmail.com (Eduardo Schettino) Date: Mon, 25 Apr 2011 23:40:18 +0800 Subject: [py-dev] [ANN] plugin: pytest-incremental Message-ID: Hi all, I have just pushed a new pytest plugin to pypi (http://pypi.python.org/pypi/pytest-incremental). The idea is to execute your tests faster by executing not all of them but only the "required" ones. This is very new (alpha), feedback is welcome :) Regards, Eduardo +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ pytest-incremental ==================== an incremental test runner (pytest plugin) What is an "incremental test runner" ? ======================================= When talking about build-tools it is common to refer to the terms: * initial (full) build - all files are compiled * incremental build (or partial rebuild) - just changed files are compiled * no-op build - no files are compiled (none changed since last execution) So an "incremental test runner" will only re-execute tests that were affected by changes in the source code since last test execution. How it works ? ================ `pytest-incremental` is a `pytest `_ plugin. So if you can run your test suite with pytest you can use `pytest-incremental`. The plugin will analyse your python source files and through its imports define the dependencies of the modules. `doit `_ is used to keep track of the dependencies and save results. The plugin will modify how pytest collect your tests. pytest do the rest of the job of actually running the tests and reporting the results. Install ========= pytest-incremental is tested on python 2.6, 2.7. ``pip install pytest-incremental``` ``python setup.py install`` local installation -------------------- You can also just grab the plugin `module `_ file and put in your project path. Then enable it (check `pytest docs `_). Usage ====== Just pass the parameter ``--incremental`` when calling from the command line:: $ py.test --incremental You can also enable it by default adding the following line to your ``pytest.ini``:: [pytest] addopts = --incremental watched packages ------------------ By default all modules collected by pytest will used as dependencies if imported. In order to limit or extend the watched folders you must use the parameter ``--watch-pkg`` Limitations ============== ``pytest-incremental`` looks for imports recursively to find dependencies (using AST). But given the very dynamic nature of python there are still some cases that a module can be affected by a module that are not detected. * `from package import *` modules imported from __all__ in a package are not counted as a dependency * modules imported not using the *import* statement * modules not explictitly imported but used at runtime (i.e. conftest.py when running your tests with pytest) * monkey-patching. (i.e. A imports X. B monkey-patches X. In this case A might depend on B) * others ? Project Details =============== - Project code + issue track on `bitbucket `_ - `Discussion group `_ From prologic at shortcircuit.net.au Wed Apr 27 01:25:49 2011 From: prologic at shortcircuit.net.au (James Mills) Date: Wed, 27 Apr 2011 09:25:49 +1000 Subject: [py-dev] [ANN] plugin: pytest-incremental In-Reply-To: References: Message-ID: On Tue, Apr 26, 2011 at 1:40 AM, Eduardo Schettino wrote: > I have just pushed a new pytest plugin to pypi > (http://pypi.python.org/pypi/pytest-incremental). > > The idea is to execute your tests faster by executing not all of them > but only the "required" ones. Nice work. I might find this useful in my own project :) cheers James -- -- James Mills -- -- "Problems are solved by method" From baptiste.lepilleur at gmail.com Wed Apr 27 22:29:20 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Wed, 27 Apr 2011 22:29:20 +0200 Subject: [py-dev] Empty traceback reported after running tests Message-ID: Hi, I'm running into the following odd output when running some tests: --- py.test --nomagic mphelpertest.py ================================================= test session starts ================================================= platform win32 -- Python 3.2.0 -- pytest-1.3.4 test path 1: mphelpertest.py mphelpertest.py ... ============================================== 3 passed in 1.16 seconds =============================================== Traceback (most recent call last): --- Notes that the traceback is empty. Any idea of what could explain that? The test is installing and (hopefully) uninstalling some logging handler and use the multiprocessing module, so it may be possible that some exceptions are thrown on shutdown when logging handler are closed (but I have not yet managed to reproduce the issue outside py.test). Sources are available here: http://gaiacrtn.free.fr/temp/pytestmp/ Baptiste. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ronny.Pfannschmidt at gmx.de Thu Apr 28 08:30:34 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Thu, 28 Apr 2011 08:30:34 +0200 Subject: [py-dev] Empty traceback reported after running tests In-Reply-To: References: Message-ID: <1303972234.30044.3.camel@Klappe2> On Wed, 2011-04-27 at 22:29 +0200, Baptiste Lepilleur wrote: > Hi, > > > I'm running into the following odd output when running some tests: > > > --- > py.test --nomagic mphelpertest.py > ================================================= test session starts > ================================================= > platform win32 -- Python 3.2.0 -- pytest-1.3.4 > test path 1: mphelpertest.py > > > mphelpertest.py ... > > > ============================================== 3 passed in 1.16 > seconds =============================================== > Traceback (most recent call last): > --- > > > Notes that the traceback is empty. Any idea of what could explain > that? > > > The test is installing and (hopefully) uninstalling some logging > handler and use the multiprocessing module, so it may be possible that > some exceptions are thrown on shutdown when logging handler are closed > (but I have not yet managed to reproduce the issue outside py.test). > can you retry with a more recent py.test afair this problem has been fixed after 1.3.4 also i would suggest to follow the py.test convention and name the test files test_{foo} instead {foo}test -- Ronny > > Sources are available here: > http://gaiacrtn.free.fr/temp/pytestmp/ > > > Baptiste. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From baptiste.lepilleur at gmail.com Thu Apr 28 09:54:45 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Thu, 28 Apr 2011 09:54:45 +0200 Subject: [py-dev] Empty traceback reported after running tests In-Reply-To: <1303972234.30044.3.camel@Klappe2> References: <1303972234.30044.3.camel@Klappe2> Message-ID: 2011/4/28 Ronny Pfannschmidt > On Wed, 2011-04-27 at 22:29 +0200, Baptiste Lepilleur wrote: > > Hi, > [...] > > > > Notes that the traceback is empty. Any idea of what could explain > > that? > > can you retry with a more recent py.test > afair this problem has been fixed after 1.3.4 > > also i would suggest to follow the py.test convention and name the test > files test_{foo} instead {foo}test > > What is the procedure for upgrading an existing install done easy_install? Do I just reinstall using the latest tarball? Or do I need to remove previous module manually (If so is it just py.test module)? Thanks, Baptiste. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ronny.Pfannschmidt at gmx.de Thu Apr 28 09:59:11 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Thu, 28 Apr 2011 09:59:11 +0200 Subject: [py-dev] Empty traceback reported after running tests In-Reply-To: References: <1303972234.30044.3.camel@Klappe2> Message-ID: <1303977551.30044.4.camel@Klappe2> On Thu, 2011-04-28 at 09:54 +0200, Baptiste Lepilleur wrote: > > > 2011/4/28 Ronny Pfannschmidt > > On Wed, 2011-04-27 at 22:29 +0200, Baptiste Lepilleur wrote: > > Hi, > [...] > > > > Notes that the traceback is empty. Any idea of what could > explain > > that? > > > can you retry with a more recent py.test > afair this problem has been fixed after 1.3.4 > > also i would suggest to follow the py.test convention and name > the test > files test_{foo} instead {foo}test > > > > What is the procedure for upgrading an existing install done > easy_install? Do I just reinstall using the latest tarball? Or do I > need to remove previous module manually (If so is it just py.test > module)? > easy_install -U pytest since easy_install wont upgrade already installed packages unless asked to -- Ronny > > Thanks, > Baptiste. > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From baptiste.lepilleur at gmail.com Thu Apr 28 19:59:01 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Thu, 28 Apr 2011 19:59:01 +0200 Subject: [py-dev] Empty traceback reported after running tests In-Reply-To: <1303977551.30044.4.camel@Klappe2> References: <1303972234.30044.3.camel@Klappe2> <1303977551.30044.4.camel@Klappe2> Message-ID: Upgrading to 2.0.3 has indeed fixed the issue (the empty traceback no longer appear). Though I don't really see an issue corresponding to that in the changelog... Thanks, Baptiste. 2011/4/28 Ronny Pfannschmidt > On Thu, 2011-04-28 at 09:54 +0200, Baptiste Lepilleur wrote: > > > > > > 2011/4/28 Ronny Pfannschmidt > > > > On Wed, 2011-04-27 at 22:29 +0200, Baptiste Lepilleur wrote: > > > Hi, > > [...] > > > > > > Notes that the traceback is empty. Any idea of what could > > explain > > > that? > > > > > > can you retry with a more recent py.test > > afair this problem has been fixed after 1.3.4 > > > > also i would suggest to follow the py.test convention and name > > the test > > files test_{foo} instead {foo}test > > > > > > > > What is the procedure for upgrading an existing install done > > easy_install? Do I just reinstall using the latest tarball? Or do I > > need to remove previous module manually (If so is it just py.test > > module)? > > > easy_install -U pytest > since easy_install wont upgrade already installed packages unless asked > to > > -- Ronny > > > > > Thanks, > > Baptiste. > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From baptiste.lepilleur at gmail.com Thu Apr 28 20:00:17 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Thu, 28 Apr 2011 20:00:17 +0200 Subject: [py-dev] Empty traceback reported after running tests In-Reply-To: <1303977551.30044.4.camel@Klappe2> References: <1303972234.30044.3.camel@Klappe2> <1303977551.30044.4.camel@Klappe2> Message-ID: Upgrading to 2.0.3 has indeed fixed the issue (the empty traceback no longer appear). Though I don't really see an issue corresponding to that in the changelog... Thanks, Baptiste. 2011/4/28 Ronny Pfannschmidt > On Thu, 2011-04-28 at 09:54 +0200, Baptiste Lepilleur wrote: > > > > > > 2011/4/28 Ronny Pfannschmidt > > > > On Wed, 2011-04-27 at 22:29 +0200, Baptiste Lepilleur wrote: > > > Hi, > > [...] > > > > > > Notes that the traceback is empty. Any idea of what could > > explain > > > that? > > > > > > can you retry with a more recent py.test > > afair this problem has been fixed after 1.3.4 > > > > also i would suggest to follow the py.test convention and name > > the test > > files test_{foo} instead {foo}test > > > > > > > > What is the procedure for upgrading an existing install done > > easy_install? Do I just reinstall using the latest tarball? Or do I > > need to remove previous module manually (If so is it just py.test > > module)? > > > easy_install -U pytest > since easy_install wont upgrade already installed packages unless asked > to > > -- Ronny > > > > > Thanks, > > Baptiste. > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ronny.Pfannschmidt at gmx.de Thu Apr 28 20:44:47 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Thu, 28 Apr 2011 20:44:47 +0200 Subject: [py-dev] Empty traceback reported after running tests In-Reply-To: References: <1303972234.30044.3.camel@Klappe2> <1303977551.30044.4.camel@Klappe2> Message-ID: <1304016287.30044.10.camel@Klappe2> On Thu, 2011-04-28 at 19:59 +0200, Baptiste Lepilleur wrote: > Upgrading to 2.0.3 has indeed fixed the issue (the empty traceback no > longer appear). > > > Though I don't really see an issue corresponding to that in the > changelog... [snip] Changes between 2.0.0 and 2.0.1 ---------------------------------------------- - refine and unify initial capturing so that it works nicely even if the logging module is used on an early-loaded conftest.py file or plugin. [snip] it's not obvious unless you know what it means. -- Ronny > Thanks, > Baptiste. > > 2011/4/28 Ronny Pfannschmidt > > On Thu, 2011-04-28 at 09:54 +0200, Baptiste Lepilleur wrote: > > > > > > 2011/4/28 Ronny Pfannschmidt > > > > On Wed, 2011-04-27 at 22:29 +0200, Baptiste > Lepilleur wrote: > > > Hi, > > [...] > > > > > > Notes that the traceback is empty. Any idea of > what could > > explain > > > that? > > > > > > can you retry with a more recent py.test > > afair this problem has been fixed after 1.3.4 > > > > also i would suggest to follow the py.test > convention and name > > the test > > files test_{foo} instead {foo}test > > > > > > > > What is the procedure for upgrading an existing install done > > easy_install? Do I just reinstall using the latest tarball? > Or do I > > need to remove previous module manually (If so is it just > py.test > > module)? > > > > easy_install -U pytest > since easy_install wont upgrade already installed packages > unless asked > to > > -- Ronny > > > > > Thanks, > > Baptiste. > > > > > > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From baptiste.lepilleur at gmail.com Fri Apr 29 10:12:26 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Fri, 29 Apr 2011 10:12:26 +0200 Subject: [py-dev] Help understanding pytest_generate_tests related error message Message-ID: Hi, I just upgraded from py.test 1.3.4 to 2.0.3, and I'm running into a new failure related to pytest_generate_tests that I can not make sense of: --- py.test src\nitroz\test_dbo2.py ================================================= test session starts ================================================= platform win32 -- Python 3.2.0 -- pytest-2.0.3 collected 0 items / 1 errors ======================================================= ERRORS ======================================================== ______________________________________ ERROR collecting src/nitroz/test_dbo2.py _______________________________________ src\nitroz\test_dbo2.py:151: in pytest_generate_tests > good_value=good_value) ) C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest\python.py:550: in addcall > pytest.fail("funcarg %r not used in this function." % name) E Failed: funcarg 'property' not used in this function. =============================================== 1 error in 0.12 seconds =============================================== --- What does this error message means? What 'this function' refer to? Source of the test module can be found there: http://pastebin.com/1D2VArGt Baptiste. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ronny.Pfannschmidt at gmx.de Fri Apr 29 12:09:24 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Fri, 29 Apr 2011 12:09:24 +0200 Subject: [py-dev] Help understanding pytest_generate_tests related error message In-Reply-To: References: Message-ID: <1304071764.30044.18.camel@Klappe2> On Fri, 2011-04-29 at 10:12 +0200, Baptiste Lepilleur wrote: > Hi, > > > I just upgraded from py.test 1.3.4 to 2.0.3, and I'm running into a > new failure related to pytest_generate_tests that I can not make sense > of: > > > --- > > > py.test src\nitroz\test_dbo2.py > ================================================= test session starts > ================================================= > platform win32 -- Python 3.2.0 -- pytest-2.0.3 > collected 0 items / 1 errors > > > ======================================================= ERRORS > ======================================================== > ______________________________________ ERROR collecting > src/nitroz/test_dbo2.py _______________________________________ > src\nitroz\test_dbo2.py:151: in pytest_generate_tests > > good_value=good_value) ) > C:\Python32\lib\site-packages\pytest-2.0.3-py3.2.egg\_pytest > \python.py:550: in addcall > > pytest.fail("funcarg %r not used in this > function." % name) > E Failed: funcarg 'property' not used in this > function. > =============================================== 1 error in 0.12 > seconds =============================================== > > > --- > > > What does this error message means? What 'this function' refer to? it seems the function is test_property_validation the current check for the metafunc funcarg dict does not consider arguments with default values funcargs thus it complains since you pass in something that has a default at the function level > > Source of the test module can be found > there: http://pastebin.com/1D2VArGt > i would like to suggest splitting the scenario types into multiple test functions, (good/bad values/types) also giving names/id's to the scenarios > > Baptiste. > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From holger at merlinux.eu Fri Apr 29 13:45:22 2011 From: holger at merlinux.eu (holger krekel) Date: Fri, 29 Apr 2011 11:45:22 +0000 Subject: [py-dev] initial take at a generic funcags hook In-Reply-To: <1303332468.8027.7.camel@Klappe2> References: <1303332468.8027.7.camel@Klappe2> Message-ID: <20110429114522.GP4071@merlinux.eu> Hi Ronny, On Wed, Apr 20, 2011 at 22:47 +0200, Ronny Pfannschmidt wrote: > hi, > > i just tried a initial implementation of generic funcargs > to solve the issue i introduced the hook > pytest_reconfigure_funcargs(request) > > the current patch is at > https://bitbucket.org/RonnyPfannschmidt/pytest-patches/qseries?apply=t&qs_apply=reconfigure-funcargs Can you detail your use case? If anything i'd rather go for a hook that would implement the whole funcarg filling protocol. This way one could use generic mechanisms (*) for forcing to execute first/last or wrapping the default protocol. I'd also like to have such a hook get called even if there are no function arguments present. It would allow to make use of request.cached_setup() with session scopes to setup global state for use by Python code. best, holger (*) you can decorate any hook impl with @pytest.mark.trylast, @pytest.mark.tryfirst and you may accept the special __multicall__ arg. these are not yet documented features but they are used in internal plugins - see there for examples. However, i guess the API is ripe to be documented and published sometime. > regards, > Ronny > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev From holger at merlinux.eu Fri Apr 29 14:00:38 2011 From: holger at merlinux.eu (holger krekel) Date: Fri, 29 Apr 2011 12:00:38 +0000 Subject: [py-dev] [ANN] plugin: pytest-incremental In-Reply-To: References: Message-ID: <20110429120038.GQ4071@merlinux.eu> Hi Eduardo, i like the idea. A few notes: * it's not compatible with pytest-xdist, is it? * i got BSDDB database corruption (i CTRL-Ced the run before) * can you add an example of a project layout and what one would call wrt to watch_pkg? I guess things don't work for me on pytest itself because it has a plugin-based dynamic namespace construction/imports so your AST scanning method does not really see the deps. A different method would be to try to record imports during the import and running of a test. Myself, i also experimented with specifying dependencies manually at some point which also solves the issue when invoking shell commands provided a project - i guess those would not naturally be found by your scanner. best, holger On Mon, Apr 25, 2011 at 23:40 +0800, Eduardo Schettino wrote: > Hi all, > > I have just pushed a new pytest plugin to pypi > (http://pypi.python.org/pypi/pytest-incremental). > > The idea is to execute your tests faster by executing not all of them > but only the "required" ones. > > This is very new (alpha), feedback is welcome :) > > Regards, > Eduardo > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > pytest-incremental > ==================== > > an incremental test runner (pytest plugin) > > > What is an "incremental test runner" ? > ======================================= > > When talking about build-tools it is common to refer to the terms: > > * initial (full) build - all files are compiled > * incremental build (or partial rebuild) - just changed files are compiled > * no-op build - no files are compiled (none changed since last execution) > > So an "incremental test runner" will only re-execute tests that were affected > by changes in the source code since last test execution. > > > How it works ? > ================ > > `pytest-incremental` is a `pytest `_ plugin. So if > you can run your test suite with pytest you can use > `pytest-incremental`. > > The plugin will analyse your python source files and through its > imports define the dependencies of the modules. `doit > `_ is used to keep track of the > dependencies and save results. The plugin will modify how pytest > collect your tests. pytest do the rest of the job of actually running > the tests and reporting the results. > > > Install > ========= > > pytest-incremental is tested on python 2.6, 2.7. > > ``pip install pytest-incremental``` > > ``python setup.py install`` > > local installation > -------------------- > > You can also just grab the plugin `module > `_ > file and put in your project path. Then enable it (check `pytest docs > `_). > > > Usage > ====== > > Just pass the parameter ``--incremental`` when calling from the command line:: > > $ py.test --incremental > > > You can also enable it by default adding the following line to your > ``pytest.ini``:: > > [pytest] > addopts = --incremental > > > watched packages > ------------------ > > By default all modules collected by pytest will used as dependencies > if imported. In order to limit or extend the watched folders you must > use the parameter ``--watch-pkg`` > > > Limitations > ============== > > ``pytest-incremental`` looks for imports recursively to find > dependencies (using AST). But given the very dynamic nature of python > there are still some cases that a module can be affected by a module > that are not detected. > > * `from package import *` modules imported from __all__ in a package > are not counted as a dependency > * modules imported not using the *import* statement > * modules not explictitly imported but used at runtime (i.e. > conftest.py when running your tests with pytest) > * monkey-patching. (i.e. A imports X. B monkey-patches X. In this > case A might depend on B) > * others ? > > > Project Details > =============== > > - Project code + issue track on `bitbucket > `_ > - `Discussion group `_ > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From Ronny.Pfannschmidt at gmx.de Fri Apr 29 15:05:04 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Fri, 29 Apr 2011 15:05:04 +0200 Subject: [py-dev] initial take at a generic funcags hook In-Reply-To: <20110429114522.GP4071@merlinux.eu> References: <1303332468.8027.7.camel@Klappe2> <20110429114522.GP4071@merlinux.eu> Message-ID: <1304082304.30044.79.camel@Klappe2> On Fri, 2011-04-29 at 11:45 +0000, holger krekel wrote: > Hi Ronny, > > On Wed, Apr 20, 2011 at 22:47 +0200, Ronny Pfannschmidt wrote: > > hi, > > > > i just tried a initial implementation of generic funcargs > > to solve the issue i introduced the hook > > pytest_reconfigure_funcargs(request) > > > > the current patch is at > > https://bitbucket.org/RonnyPfannschmidt/pytest-patches/qseries?apply=t&qs_apply=reconfigure-funcargs > > Can you detail your use case? > in the setup for the bitbucket hook tests of pypy i have a little mess of interacting funcargs (im intercepting various calls by default) that also needs some additional funcargs to functions that shouldn't need them so having a generic hook to set the interaction between up would be helpful > If anything i'd rather go for a hook that would implement the whole > funcarg filling protocol. This way one could use generic mechanisms (*) > for forcing to execute first/last or wrapping the default protocol. > I'd also like to have such a hook get called even if there are no > function arguments present. It would allow to make use of > request.cached_setup() with session scopes to setup global state > for use by Python code. > i suppose a pytest_configure_funcargs(request) hook should do the builtin implementation would be a tryfirst, my use of "reconfigure" would be the same -- Ronny > best, > holger > > (*) you can decorate any hook impl with @pytest.mark.trylast, > @pytest.mark.tryfirst and you may accept the special __multicall__ arg. > these are not yet documented features but they are used in internal > plugins - see there for examples. However, i guess the API is ripe > to be documented and published sometime. > > > > regards, > > Ronny > > > > > _______________________________________________ > > py-dev mailing list > > py-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/py-dev > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From schettino72 at gmail.com Fri Apr 29 15:54:27 2011 From: schettino72 at gmail.com (Eduardo Schettino) Date: Fri, 29 Apr 2011 21:54:27 +0800 Subject: [py-dev] [ANN] plugin: pytest-incremental In-Reply-To: <20110429120038.GQ4071@merlinux.eu> References: <20110429120038.GQ4071@merlinux.eu> Message-ID: On Fri, Apr 29, 2011 at 8:00 PM, holger krekel wrote: > * it's not compatible with pytest-xdist, is it? I actually had never tried pytest-xdist... is there anything that I could do to make them compatible? > * i got BSDDB database corruption (i CTRL-Ced the run before) I tried hitting CTRL-C at several points and never got a corruption. At which point you hit Ctrl-C? before the test execution starts? Although I got a bug that it does not detect that not all tests were executed and mark them as successful. > * can you add an example of a project layout The plugin is supposed to work with any project layout... > and what one would call wrt to watch_pkg? By default it will look for changes in all python modules that pass through py.test collection. This way doesnt work well when you try to run tests from a single file like: $ py.test tests/test_foo.py If you try to use the plugin like this it will give an error message saying that you must specify watch_pkg. lets say you have the folders: /tests /my_lib you should call $ py.test --incremental --watch-pkg my_lib tests/test_foo.py (no need to pass the package of the test file itself) It can also be used in case you want to watch for changes in modules that are in another project. for example if you are testing pytest and want to check for changes in dependencies from your "py" package. $ py.test --incremental --watch-pkg my_lib --watch-pkg ../py-trunk/py > > I guess things don't work for me on pytest itself because > it has a plugin-based dynamic namespace construction/imports > so your AST scanning method does not really see the deps. > A different method would be to try to record imports > during the import and running of a test. ?Myself, i also > experimented with specifying dependencies manually at > some point which also solves the issue when invoking > shell commands provided a project - i guess those > would not naturally be found by your scanner. > Yes. There is also the problem of dependencies on text files (or any other non-python files). I think dependencies should really be defined by the user, this AST scanner should be just one way of doing it that works out of the box for most projects. Regards, Eduardo From holger at merlinux.eu Fri Apr 29 20:05:55 2011 From: holger at merlinux.eu (holger krekel) Date: Fri, 29 Apr 2011 18:05:55 +0000 Subject: [py-dev] [ANN] plugin: pytest-incremental In-Reply-To: References: <20110429120038.GQ4071@merlinux.eu> Message-ID: <20110429180555.GU4071@merlinux.eu> On Fri, Apr 29, 2011 at 21:54 +0800, Eduardo Schettino wrote: > On Fri, Apr 29, 2011 at 8:00 PM, holger krekel wrote: > > * it's not compatible with pytest-xdist, is it? > I actually had never tried pytest-xdist... is there anything that I > could do to make them compatible? Many people use it for distributing tests to multiple CPUs (or hosts). If you just consider the multi-CPU case the main issue is to make sure the slave processes don't step onto each other when writing or determining your state information. > > * i got BSDDB database corruption (i CTRL-Ced the run before) > I tried hitting CTRL-C at several points and never got a corruption. > At which point you hit Ctrl-C? before the test execution starts? > Although I got a bug that it does not detect that not all tests were > executed and mark them as successful. Not sure i can help with reproducing it. > > * can you add an example of a project layout > The plugin is supposed to work with any project layout... ok ... > > and what one would call wrt to watch_pkg? > By default it will look for changes in all python modules that pass > through py.test collection. This way doesnt work well when you try to > run tests from a single file like: > $ py.test tests/test_foo.py > > If you try to use the plugin like this it will give an error message > saying that you must specify watch_pkg. lets say you have the folders: > /tests > /my_lib > > you should call > $ py.test --incremental --watch-pkg my_lib tests/test_foo.py > (no need to pass the package of the test file itself) > > It can also be used in case you want to watch for changes in modules > that are in another project. for example if you are testing pytest and > want to check for changes in dependencies from your "py" package. > $ py.test --incremental --watch-pkg my_lib --watch-pkg ../py-trunk/py ok, now it's clear that one need to specify file system paths. Throughout the python world "--XYZ-pkg" might mean a module import path like "os.path" or a filesystem path. Maybe good to mention this in the help string for the option - i'd probably rather call it "--watch-path" to disambiguate. > > I guess things don't work for me on pytest itself because > > it has a plugin-based dynamic namespace construction/imports > > so your AST scanning method does not really see the deps. > > A different method would be to try to record imports > > during the import and running of a test. ?Myself, i also > > experimented with specifying dependencies manually at > > some point which also solves the issue when invoking > > shell commands provided a project - i guess those > > would not naturally be found by your scanner. > > > Yes. There is also the problem of dependencies on text files (or any > other non-python files). > I think dependencies should really be defined by the user, this AST > scanner should be just one way of doing it that works out of the box > for most projects. Right. Do you plan to implement a manual way to specify deps for your plugin? sidenote: you may want to announce the next release of the plugin also to the TIP (testing in python) list - http://lists.idyll.org/listinfo/testing-in-python - a number of people are rather following there rather than here. best, holger > Regards, > Eduardo > From baptiste.lepilleur at gmail.com Fri Apr 29 22:18:20 2011 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Fri, 29 Apr 2011 22:18:20 +0200 Subject: [py-dev] Help understanding pytest_generate_tests related error message In-Reply-To: <1304071764.30044.18.camel@Klappe2> References: <1304071764.30044.18.camel@Klappe2> Message-ID: 2011/4/29 Ronny Pfannschmidt > [...] > > E Failed: funcarg 'property' not used in this > > function. > [...] > it seems the function is test_property_validation > the current check for the metafunc funcarg dict does not consider > arguments with default values funcargs > thus it complains since you pass in something that has a default at the > function level > It was indeed the cause. Changing: def test_property_validation( validation_scenario_iter, property=None, value=None, good_value=None ): to: def test_property_validation( validation_scenario_iter, property, value, good_value ): fixed the issue. This is weird as it used to work in 1.3.4... > > > > Source of the test module can be found > > there: http://pastebin.com/1D2VArGt > > > > i would like to suggest splitting the scenario types into multiple test > functions, (good/bad values/types) > > I'm not sure how I would go about that: it would means going through the VALIDATION_SCENARIOS test data for each test function, and duplicating some setup code of the test function... also giving names/id's to the scenarios I did not know it was possible to give name. I added this and it greatly improve the output! Thanks a lot for helping figuring this out, Baptiste. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ronny.Pfannschmidt at gmx.de Fri Apr 29 22:25:04 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Fri, 29 Apr 2011 22:25:04 +0200 Subject: [py-dev] Help understanding pytest_generate_tests related error message In-Reply-To: References: <1304071764.30044.18.camel@Klappe2> Message-ID: <1304108704.30044.81.camel@Klappe2> On Fri, 2011-04-29 at 22:18 +0200, Baptiste Lepilleur wrote: > 2011/4/29 Ronny Pfannschmidt > > [...] > > E Failed: funcarg 'property' not used in > this > > function. > [...] > it seems the function is test_property_validation > the current check for the metafunc funcarg dict does not > consider > arguments with default values funcargs > thus it complains since you pass in something that has a > default at the > function level > > > It was indeed the cause. Changing: > def test_property_validation( validation_scenario_iter, property=None, > value=None, good_value=None ): > > > to: > > > def test_property_validation( validation_scenario_iter, property, > value, good_value ): > > > fixed the issue. This is weird as it used to work in 1.3.4... > > > > > Source of the test module can be found > > there: http://pastebin.com/1D2VArGt > > > > > i would like to suggest splitting the scenario types into > multiple test > functions, (good/bad values/types) > > > > I'm not sure how I would go about that: it would means going through > the VALIDATION_SCENARIOS test data for each test function, and > duplicating some setup code of the test function... unless i'm misstaken, only the creation of the class is shared which means that could actually be something like a funcarg or a param to a funcarg -- Ronny > > > also giving names/id's to the scenarios > I did not know it was possible to give name. I added this and it > greatly improve the output! > > > Thanks a lot for helping figuring this out, > Baptiste. > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From holger at merlinux.eu Sat Apr 30 16:22:25 2011 From: holger at merlinux.eu (holger krekel) Date: Sat, 30 Apr 2011 14:22:25 +0000 Subject: [py-dev] migration from 1.3 to 2.0 In-Reply-To: References: <20110405201612.GW16231@merlinux.eu> <20110407061715.GE16231@merlinux.eu> Message-ID: <20110430142225.GY4071@merlinux.eu> On Thu, Apr 07, 2011 at 12:29 -0400, Vyacheslav Rafalskiy wrote: > >> >> 3. In my pytest_configure() there are numerous conditions when it can > >> >> fail. For this > >> >> conditions I have exceptions with specially crafted messages, intended > >> >> for different > >> >> people. Now they are gone, replaced by a long and scary listing with every line > >> >> prepended with INTERNALERROR. What is internal about it? Can I continue managing > >> >> my configuration errors? > >> > > >> > Sure, you should be able to. I guess it was a bit of an incident that > >> > failures in pytest_configure were shown nicely. ?I am not quite sure > >> > immediately what the best way to relay "global" configuration messages > >> > to the user is. ?If you'd use "funcargs" and the "session" scope for > >> > setting up resources then it would show nicely. ?Feel free to open > >> > a feature/enhancement request to have py.test show failures > >> > in sessionstart or configure hooks in a way that helps you. > >> > This way we can write a specific test and make sure it works > >> > also in the future. > >> > >> If I understand correctly, using funcargs means that every test function > >> of hundreds I have in several suites should include a reference to the > >> global setup funcarg. It seems a non-starter. > >> > >> I guess I will stick to the old version and try to think of something > >> ?to enter in a feature request. > > > > sure, makes sense. ?isn't your main issue that upon actual test start > > (after collection) you want to do some configuration steps which might > > result in error conditions which you'd like to see nicely reported to > > the user? > > That's exactly my point. > > >(sidenote: configure and even sessionstart hooks are both a bit > > not 100% right because they happen even on the master side of a distributed > > test run and the master side does not collect or run tests at all) > > I see. Perhaps something like setup_package() in the top-level __init__.py > could be a solution? I guess you mean an __init__.py file of a test directory. With a layout of test dirs within an application this might mean one has to put this setup into the main package __init__.py and mixing test code and application code is often not a good idea. So i'd rather put it into a conftest.py file as a normal hook. Maybe "pytest_pyfunc_setup(request)" would be good where request is the same object as for the funcarg factories. You could then write: # content of conftest.py def pytest_pyfunc_setup(request): val = request.cached_setup(setup=makeval, scope="session") # use val for some global setting of the package Alternatively we could see to call something like: def pytest_setup_testloop(config): val = makeval() # use val for some global setting of the package def pytest_teardown_testloop(config): ... which would be called once for a test process. best, holger > Vyacheslav > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev >