From holger at merlinux.eu Wed Aug 1 15:18:02 2012 From: holger at merlinux.eu (holger krekel) Date: Wed, 1 Aug 2012 13:18:02 +0000 Subject: [py-dev] new resource API nearing completion including impl Message-ID: <20120801131802.GY26019@merlinux.eu> Hi all, I've just uploaded new docs and my latest changes to the newstyle funcargs and setup mechanism. I am getting increasingly happy about it and think i solved most of the implementation problems now. I also moved to writing direct release-relevant documentation now. The new main doc is http://pytest.org/dev/resources.html It contains a number of working examples. To play yourself you can use the repo or use "pip install -i http://pypi.testrun.org -U pytest". Mostly due to implementation but also for communication reasons i resolved to advertising a "newstyle" API that does not fully include the oldstyle API. Mainly you cannot use the "request" object in setup/factory-marked functions. Intstead there is a more minimal "testcontext" object which specifically does not offer cached_setup/getfuncargvalue because you can and should now directly use funcargs in factory/setup functions. Nevertheless, old-style and new-style resources can use each other and i think i managed full backward compatibility. If i see it correctly there is at least one open issue that was originally raised by Floris that is not covered yet: How to write a setup-function that instantiates an object depending on the presence of a "@needsdb" marker. I have several ideas about this but probably it's best to discuss this on IRC ... best, holger From flub at devork.be Thu Aug 2 00:21:52 2012 From: flub at devork.be (Floris Bruynooghe) Date: Wed, 1 Aug 2012 23:21:52 +0100 Subject: [py-dev] new resource API nearing completion including impl In-Reply-To: <20120801131802.GY26019@merlinux.eu> References: <20120801131802.GY26019@merlinux.eu> Message-ID: Hello Holger, Apologies for not responding earlier, but I've been on holiday. In general this looks like it is shaping up rather nicely. My first part of feedback is somewhat bikeshedding-like: while using the pytest.mark implementation makes a lot of sense I do wonder whether it does not make more sense to keep the pytest.mark api for marking test functions and not to start using it for setup/factory functions. The setup functions would be equally clear when written as @pytest.factory, @pytest.setup etc and I think keeping the way of marking test functions with setup/factory functions different is worthwhile. Secondly, and this could be a bad idea, while I do like the new decorators I did grow attached to the old pytest_funcarg__* syntax even if it could be argued they are a bit more magical. Since there already is a precedent for using __tracebackhide__ I was wondering if the scoping could be added to the old-style funcargs using e.g. __scope__ and maybe even __parameterise__ in the function body? Old-style funcargs could also be made to directly accept other funcargs/resources I think and so really bridge the gap. I do realise however that this would probably seem pretty weird to the general python public and decorators are probably a better api, but I still wanted to mention this. A better idea is probably marking pytest_funcarg__* functions with @factory but I'm failing to use the new-style resources code for now so not sure if that works. The setting and parameterisation of a global in the introduction of @pytest.mark.setup seems very advanced and not very suitable to introduce the @setup decorator. I'm actually rather dubious to it's use, it seems very difficult to notice that the test_1 and test_2 functions will be invoked twice. While it is very nice for xUnit setup functions to have access to funcargs/resources I'm not actually convinced the decorator version adds much value, they already have an explicit and well-known scope associated with them via their location. If there really is a need for modifying the scope or adding parametrisation (which I'm not sure about, I think funcargs/resouces could achieve the same in a more obvious way) then just re-using @factory on the existing xUnit seems like less confusing approach. I hope this feedback makes sense so far, I apologise if not, I'm pretty tired right now. I'd really like to have a go at making a prototype of pytest-django using this in order to give more feedback, but that's not for tonight. There are a few interesting cases I encountered there which I should try out and I'm intrigued to see if the parametrisation would allow it to test multiple db backends in one process (probably not, but that will be Django's fault, not py.test's). Regards, Floris On 1 August 2012 14:18, holger krekel wrote: > Hi all, > > I've just uploaded new docs and my latest changes to the newstyle funcargs > and setup mechanism. I am getting increasingly happy about it and think > i solved most of the implementation problems now. I also moved to > writing direct release-relevant documentation now. The new main doc is > > http://pytest.org/dev/resources.html > > It contains a number of working examples. To play yourself you can use > the repo or use "pip install -i http://pypi.testrun.org -U pytest". > > Mostly due to implementation but also for communication reasons > i resolved to advertising a "newstyle" API that does not fully include > the oldstyle API. Mainly you cannot use the "request" object in > setup/factory-marked functions. Intstead there is a more minimal > "testcontext" object which specifically does not offer > cached_setup/getfuncargvalue because you can and should now directly use > funcargs in factory/setup functions. Nevertheless, old-style and > new-style resources can use each other and i think i managed full > backward compatibility. > > If i see it correctly there is at least one open issue that was > originally raised by Floris that is not covered yet: How to write a > setup-function that instantiates an object depending on the presence of > a "@needsdb" marker. I have several ideas about this but probably it's > best to discuss this on IRC ... > > best, > > holger > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev From holger at merlinux.eu Thu Aug 2 09:36:39 2012 From: holger at merlinux.eu (holger krekel) Date: Thu, 2 Aug 2012 07:36:39 +0000 Subject: [py-dev] new resource API nearing completion including impl In-Reply-To: References: <20120801131802.GY26019@merlinux.eu> Message-ID: <20120802073639.GZ26019@merlinux.eu> Hi Floris, On Wed, Aug 01, 2012 at 23:21 +0100, Floris Bruynooghe wrote: > Hello Holger, > > Apologies for not responding earlier, but I've been on holiday. You are just-in-time right for this. It anyway took me a while to sort out impl issues. > In general this looks like it is shaping up rather nicely. > > My first part of feedback is somewhat bikeshedding-like: while using > the pytest.mark implementation makes a lot of sense I do wonder > whether it does not make more sense to keep the pytest.mark api for > marking test functions and not to start using it for setup/factory > functions. The setup functions would be equally clear when written as > @pytest.factory, @pytest.setup etc and I think keeping the way of > marking test functions with setup/factory functions different is > worthwhile. Good point! Apart from confusingness, The features of applying marks to classes or modules are of no use here. Making them pytest.X functions also allows a more concise and better documented signature. Very good bikeshedding! :) > Secondly, and this could be a bad idea, while I do like the new > decorators I did grow attached to the old pytest_funcarg__* syntax > even if it could be argued they are a bit more magical. Since there > already is a precedent for using __tracebackhide__ I was wondering if > the scoping could be added to the old-style funcargs using e.g. > __scope__ and maybe even __parameterise__ in the function body? Not really possible i think. Traceback-printing has the frames at hand to look at locals(). When the namespace of a module is scanned the bodies of functions are opaque. > Old-style funcargs could also be made to directly accept other > funcargs/resources I think and so really bridge the gap. Didn't document it but old-style funcargs do accept other funcargs actually. > I do realise however that this would probably seem pretty weird to the > general python public and decorators are probably a better api, but I > still wanted to mention this. My goal when designing the last incarnation of the API was to make it easy for newcomers. > A better idea is probably marking pytest_funcarg__* functions with > @factory but I'm failing to use the new-style resources code for now > so not sure if that works. Why does it not work for you? (Ping me on IRC maybe). > > The setting and parameterisation of a global in the introduction of > @pytest.mark.setup seems very advanced and not very suitable to > introduce the @setup decorator. Good point again. I was focused on the difficult cases first - the @setup documentation is utterly lacking, sorry about that. Probably it should go to its own page and fully document all xunit functionality. > I'm actually rather dubious to it's > use, it seems very difficult to notice that the test_1 and test_2 > functions will be invoked twice. While it is very nice for xUnit > setup functions to have access to funcargs/resources I'm not actually > convinced the decorator version adds much value, they already have an > explicit and well-known scope associated with them via their location. Maybe i am wrong but I guess you got this impression because i didn't document the wide range of uses of the setup functions. It's actually a feature that you can define per-function/per-class/per-module setup code in conftest.py files or plugins. It is to replace almost all uses of implementing pytest_runtest_init/pytest_sessionstart. > If there really is a need for modifying the scope or adding > parametrisation (which I'm not sure about, I think funcargs/resouces > could achieve the same in a more obvious way) then just re-using > @factory on the existing xUnit seems like less confusing approach. The main difference between setup and factory/funcargs is that setup performs side-effects so test functions do not need to list funcargs in their signature. > I hope this feedback makes sense so far, I apologise if not, I'm > pretty tired right now. I'd really like to have a go at making a > prototype of pytest-django using this in order to give more feedback, > but that's not for tonight. There are a few interesting cases I > encountered there which I should try out and I'm intrigued to see if > the parametrisation would allow it to test multiple db backends in one > process (probably not, but that will be Django's fault, not > py.test's). Your feedback as always is very valuable, thanks for taking the time! As to Django, maybe Carl can help by stating his guess if it's possible at all to successively instantiate Django with different backends within one process. best, holger > > Regards, > Floris > > > On 1 August 2012 14:18, holger krekel wrote: > > Hi all, > > > > I've just uploaded new docs and my latest changes to the newstyle funcargs > > and setup mechanism. I am getting increasingly happy about it and think > > i solved most of the implementation problems now. I also moved to > > writing direct release-relevant documentation now. The new main doc is > > > > http://pytest.org/dev/resources.html > > > > It contains a number of working examples. To play yourself you can use > > the repo or use "pip install -i http://pypi.testrun.org -U pytest". > > > > Mostly due to implementation but also for communication reasons > > i resolved to advertising a "newstyle" API that does not fully include > > the oldstyle API. Mainly you cannot use the "request" object in > > setup/factory-marked functions. Intstead there is a more minimal > > "testcontext" object which specifically does not offer > > cached_setup/getfuncargvalue because you can and should now directly use > > funcargs in factory/setup functions. Nevertheless, old-style and > > new-style resources can use each other and i think i managed full > > backward compatibility. > > > > If i see it correctly there is at least one open issue that was > > originally raised by Floris that is not covered yet: How to write a > > setup-function that instantiates an object depending on the presence of > > a "@needsdb" marker. I have several ideas about this but probably it's > > best to discuss this on IRC ... > > > > best, > > > > holger > > _______________________________________________ > > py-dev mailing list > > py-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/py-dev > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From holger at merlinux.eu Thu Aug 2 12:44:53 2012 From: holger at merlinux.eu (holger krekel) Date: Thu, 2 Aug 2012 10:44:53 +0000 Subject: [py-dev] new resource API nearing completion including impl In-Reply-To: <20120802073639.GZ26019@merlinux.eu> References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> Message-ID: <20120802104453.GC26019@merlinux.eu> Hi Again, On Thu, Aug 02, 2012 at 07:36 +0000, holger krekel wrote: > Hi Floris, > > On Wed, Aug 01, 2012 at 23:21 +0100, Floris Bruynooghe wrote: > > Hello Holger, > > > > Apologies for not responding earlier, but I've been on holiday. > > You are just-in-time right for this. It anyway took me a while > to sort out impl issues. > > > In general this looks like it is shaping up rather nicely. > > > > My first part of feedback is somewhat bikeshedding-like: while using > > the pytest.mark implementation makes a lot of sense I do wonder > > whether it does not make more sense to keep the pytest.mark api for > > marking test functions and not to start using it for setup/factory > > functions. The setup functions would be equally clear when written as > > @pytest.factory, @pytest.setup etc and I think keeping the way of > > marking test functions with setup/factory functions different is > > worthwhile. > > Good point! Apart from confusingness, The features of applying marks to > classes or modules are of no use here. Making them pytest.X functions > also allows a more concise and better documented signature. Very good > bikeshedding! :) I've now implemented the move to @pytest.factory and @pytest.setup and also updated and refined the documentation a bit, see: http://pytest.org/dev/resources.html and the new http://pytest.org/dev/setup.html Hope the latter begins to make more sense. I still intend to refine docs and add more examples but now is lunch and then child-care summer party time :) I also uploaded a new package pytest-2.3.0.dev8 to be installed via: pip install -i http://pypi.testrun.org -U pytest best, holger > > Secondly, and this could be a bad idea, while I do like the new > > decorators I did grow attached to the old pytest_funcarg__* syntax > > even if it could be argued they are a bit more magical. Since there > > already is a precedent for using __tracebackhide__ I was wondering if > > the scoping could be added to the old-style funcargs using e.g. > > __scope__ and maybe even __parameterise__ in the function body? > > Not really possible i think. Traceback-printing has the frames at > hand to look at locals(). When the namespace of a module is scanned > the bodies of functions are opaque. > > > Old-style funcargs could also be made to directly accept other > > funcargs/resources I think and so really bridge the gap. > > Didn't document it but old-style funcargs do accept other funcargs actually. > > > I do realise however that this would probably seem pretty weird to the > > general python public and decorators are probably a better api, but I > > still wanted to mention this. > > My goal when designing the last incarnation of the API was to make it > easy for newcomers. > > > A better idea is probably marking pytest_funcarg__* functions with > > @factory but I'm failing to use the new-style resources code for now > > so not sure if that works. > > Why does it not work for you? (Ping me on IRC maybe). > > > > > The setting and parameterisation of a global in the introduction of > > @pytest.mark.setup seems very advanced and not very suitable to > > introduce the @setup decorator. > > Good point again. I was focused on the difficult cases first - > the @setup documentation is utterly lacking, sorry about that. > Probably it should go to its own page and fully document all xunit > functionality. > > > I'm actually rather dubious to it's > > use, it seems very difficult to notice that the test_1 and test_2 > > functions will be invoked twice. While it is very nice for xUnit > > setup functions to have access to funcargs/resources I'm not actually > > convinced the decorator version adds much value, they already have an > > explicit and well-known scope associated with them via their location. > > Maybe i am wrong but I guess you got this impression because i > didn't document the wide range of uses of the setup functions. It's > actually a feature that you can define per-function/per-class/per-module > setup code in conftest.py files or plugins. It is to replace almost all > uses of implementing pytest_runtest_init/pytest_sessionstart. > > > If there really is a need for modifying the scope or adding > > parametrisation (which I'm not sure about, I think funcargs/resouces > > could achieve the same in a more obvious way) then just re-using > > @factory on the existing xUnit seems like less confusing approach. > > The main difference between setup and factory/funcargs is that setup > performs side-effects so test functions do not need to list funcargs in > their signature. > > > I hope this feedback makes sense so far, I apologise if not, I'm > > pretty tired right now. I'd really like to have a go at making a > > prototype of pytest-django using this in order to give more feedback, > > but that's not for tonight. There are a few interesting cases I > > encountered there which I should try out and I'm intrigued to see if > > the parametrisation would allow it to test multiple db backends in one > > process (probably not, but that will be Django's fault, not > > py.test's). > > Your feedback as always is very valuable, thanks for taking the time! > As to Django, maybe Carl can help by stating his guess if it's possible at all > to successively instantiate Django with different backends within > one process. > > best, > holger > > > > > Regards, > > Floris > > > > > > On 1 August 2012 14:18, holger krekel wrote: > > > Hi all, > > > > > > I've just uploaded new docs and my latest changes to the newstyle funcargs > > > and setup mechanism. I am getting increasingly happy about it and think > > > i solved most of the implementation problems now. I also moved to > > > writing direct release-relevant documentation now. The new main doc is > > > > > > http://pytest.org/dev/resources.html > > > > > > It contains a number of working examples. To play yourself you can use > > > the repo or use "pip install -i http://pypi.testrun.org -U pytest". > > > > > > Mostly due to implementation but also for communication reasons > > > i resolved to advertising a "newstyle" API that does not fully include > > > the oldstyle API. Mainly you cannot use the "request" object in > > > setup/factory-marked functions. Intstead there is a more minimal > > > "testcontext" object which specifically does not offer > > > cached_setup/getfuncargvalue because you can and should now directly use > > > funcargs in factory/setup functions. Nevertheless, old-style and > > > new-style resources can use each other and i think i managed full > > > backward compatibility. > > > > > > If i see it correctly there is at least one open issue that was > > > originally raised by Floris that is not covered yet: How to write a > > > setup-function that instantiates an object depending on the presence of > > > a "@needsdb" marker. I have several ideas about this but probably it's > > > best to discuss this on IRC ... > > > > > > best, > > > > > > holger > > > _______________________________________________ > > > py-dev mailing list > > > py-dev at codespeak.net > > > http://codespeak.net/mailman/listinfo/py-dev > > _______________________________________________ > > py-dev mailing list > > py-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/py-dev > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From Ronny.Pfannschmidt at gmx.de Thu Aug 2 13:32:20 2012 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Thu, 02 Aug 2012 13:32:20 +0200 Subject: [py-dev] new resource API nearing completion including impl In-Reply-To: <20120801131802.GY26019@merlinux.eu> References: <20120801131802.GY26019@merlinux.eu> Message-ID: <501A6544.1080306@gmx.de> Hi Holger, i don't see a way to determine parametrization of a global resource at pytest_configure time (or later) it would be nice to be able to determine them after configure for considering cli args -- Ronny From holger at merlinux.eu Thu Aug 2 14:22:48 2012 From: holger at merlinux.eu (holger krekel) Date: Thu, 2 Aug 2012 12:22:48 +0000 Subject: [py-dev] new resource API nearing completion including impl In-Reply-To: <501A6544.1080306@gmx.de> References: <20120801131802.GY26019@merlinux.eu> <501A6544.1080306@gmx.de> Message-ID: <20120802122248.GD26019@merlinux.eu> Hi Ronny, On Thu, Aug 02, 2012 at 13:32 +0200, Ronny Pfannschmidt wrote: > Hi Holger, > > i don't see a way to determine parametrization of a global resource > at pytest_configure time (or later) > > it would be nice to be able to determine them after configure for > considering cli args I was thinking about allowing a function for "params": @pytest.factory(params=func) That function would receive the "config" object and maybe some marker information. (It would be called at collection time.) best, holger > -- Ronny > From flub at devork.be Thu Aug 2 14:50:50 2012 From: flub at devork.be (Floris Bruynooghe) Date: Thu, 2 Aug 2012 13:50:50 +0100 Subject: [py-dev] new resource API nearing completion including impl In-Reply-To: <20120802104453.GC26019@merlinux.eu> References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> Message-ID: On 2 August 2012 11:44, holger krekel wrote: > http://pytest.org/dev/setup.html > > Hope the latter begins to make more sense. Yes, it does. I now see the power @setup. One thing you might want to add is compare the module-global setting to simply using the "global" statement inside the setup function. Btw, is it a bug in the assertion that when using a global variable the assert-printing does not seem to show the value of that global variable? > I still intend to refine docs and add more examples but now is lunch and > then child-care summer party time :) I hope you're having a better summer to party in then the torrential rain we seem to be getting this afternoon ;-) > I also uploaded a new package pytest-2.3.0.dev8 to be installed > via: > > pip install -i http://pypi.testrun.org -U pytest I was playing with this over lunch and discovered this doesn't work: @pytest.factory(scope='session') def pytest_funcarg__foo(): return 42 Would it not make sense to allow this (or at least provide a clearer error)? I still like that form because of the grep-ability (doing a 2-line grep is much harder and would still not cover ppl doing "from pytest import factory" etc). Also doing this results in setup_module being called twice: @pytest.setup(scope='module') def setup_module(): print 'setting up module' I'm not sure what the correct behaviour should be here. Regards, Floris From Ronny.Pfannschmidt at gmx.de Thu Aug 2 14:54:38 2012 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Thu, 02 Aug 2012 14:54:38 +0200 Subject: [py-dev] new resource API nearing completion including impl In-Reply-To: <20120802122248.GD26019@merlinux.eu> References: <20120801131802.GY26019@merlinux.eu> <501A6544.1080306@gmx.de> <20120802122248.GD26019@merlinux.eu> Message-ID: <501A788E.8050203@gmx.de> Hi Holger, > > I was thinking about allowing a function for "params": > > @pytest.factory(params=func) > > That function would receive the "config" object and maybe some > marker information. (It would be called at collection time.) > sounds good, btw, what about other parameters that could/should be global for example in anyvc i got sets of xspecs that get feed into a different ressource but they dont really have other code around them, should i just pass on testressource.param as result? -- Ronny From holger at merlinux.eu Thu Aug 2 19:19:42 2012 From: holger at merlinux.eu (holger krekel) Date: Thu, 2 Aug 2012 17:19:42 +0000 Subject: [py-dev] new resource API nearing completion including impl In-Reply-To: <501A788E.8050203@gmx.de> References: <20120801131802.GY26019@merlinux.eu> <501A6544.1080306@gmx.de> <20120802122248.GD26019@merlinux.eu> <501A788E.8050203@gmx.de> Message-ID: <20120802171942.GE26019@merlinux.eu> On Thu, Aug 02, 2012 at 14:54 +0200, Ronny Pfannschmidt wrote: > Hi Holger, > > > > >I was thinking about allowing a function for "params": > > > > @pytest.factory(params=func) > > > >That function would receive the "config" object and maybe some > >marker information. (It would be called at collection time.) > > > sounds good, > > btw, what about other parameters that could/should be global > > for example in anyvc i got sets of xspecs > that get feed into a different ressource > but they dont really have other code around them, > should i just pass on testressource.param as result? I guess so. Not sure i understand "parameters that should be global". best, holger > -- Ronny > From holger at merlinux.eu Thu Aug 2 19:24:19 2012 From: holger at merlinux.eu (holger krekel) Date: Thu, 2 Aug 2012 17:24:19 +0000 Subject: [py-dev] new resource API nearing completion including impl In-Reply-To: References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> Message-ID: <20120802172419.GF26019@merlinux.eu> On Thu, Aug 02, 2012 at 13:50 +0100, Floris Bruynooghe wrote: > On 2 August 2012 11:44, holger krekel wrote: > > http://pytest.org/dev/setup.html > > > > Hope the latter begins to make more sense. > > Yes, it does. I now see the power @setup. One thing you might want > to add is compare the module-global setting to simply using the > "global" statement inside the setup function. Do you mean that in the case where the global-setting happens in the conftest.py using "global" does not work? > Btw, is it a bug in the assertion that when using a global variable > the assert-printing does not seem to show the value of that global > variable? I noticed this as well and consider it a bug, yes. > > I still intend to refine docs and add more examples but now is lunch and > > then child-care summer party time :) > > I hope you're having a better summer to party in then the torrential > rain we seem to be getting this afternoon ;-) Yes, almost too hot here actually ... > > I also uploaded a new package pytest-2.3.0.dev8 to be installed > > via: > > > > pip install -i http://pypi.testrun.org -U pytest > > I was playing with this over lunch and discovered this doesn't work: > > @pytest.factory(scope='session') > def pytest_funcarg__foo(): > return 42 > > Would it not make sense to allow this (or at least provide a clearer > error)? I still like that form because of the grep-ability (doing a > 2-line grep is much harder and would still not cover ppl doing "from > pytest import factory" etc). Grepability is an argument. Would adding a "name=..." parameter for the factory-decorator help enough? Allowing and advertising pytest_funcarg__foo feels strange to somehow taking a fresh look i think. > Also doing this results in setup_module being called twice: > > @pytest.setup(scope='module') > def setup_module(): > print 'setting up module' > > I'm not sure what the correct behaviour should be here. Hum, I think the decorator "consumes" the function and it should not be considered for anything else. Do you agree? best, holger > > Regards, > Floris > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From flub at devork.be Thu Aug 2 20:47:21 2012 From: flub at devork.be (Floris Bruynooghe) Date: Thu, 2 Aug 2012 19:47:21 +0100 Subject: [py-dev] new resource API nearing completion including impl In-Reply-To: <20120802172419.GF26019@merlinux.eu> References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> <20120802172419.GF26019@merlinux.eu> Message-ID: On 2 August 2012 18:24, holger krekel wrote: > On Thu, Aug 02, 2012 at 13:50 +0100, Floris Bruynooghe wrote: >> On 2 August 2012 11:44, holger krekel wrote: >> > http://pytest.org/dev/setup.html >> > >> > Hope the latter begins to make more sense. >> >> Yes, it does. I now see the power @setup. One thing you might want >> to add is compare the module-global setting to simply using the >> "global" statement inside the setup function. > > Do you mean that in the case where the global-setting happens in the > conftest.py using "global" does not work? They both work, I meant: @pytest.setup(scope='module') def mysetup(): global myvariable myvariable = 42 vs: @pytest.setup(scope='module') def mysetup(testcontext): testcontext.module.myvariable = 42 Intuitively I would go for the first option and I think most python developers would as well since it is what they know. I know the second option, which you demonstrate, is possible from inside a plugin or a conftest.py but I probably would consider this a bad practice since they are not logical places to inject globals. Actually injecting globals seems weird to me, I used to do this when having plain old unittest but ever since funcargs I just type in the argument and find that much clearer. So in short, while it is nice you can do this I don't think it is a great example as, IMHO, it encourages a bad practice. Would a better example be checking for a mark on the function and doing some special setup based on that? Maybe the special setup is requesting another funcarg/resource? (But there doesn't seem to be an API on the testcontext to request a custom funcarg/resource. Is this meant to be disallowed?) Which brings me onto the next topic, the testcontext object provides access to .function, which you can use to check marks etc. But when you had merged items/requests we had access to the item and you where even talking about having item.applymarker() and a specific API for inspecting markers. Would it not be nice to have these APIs on testcontext (if the sope is "function")? >> Btw, is it a bug in the assertion that when using a global variable >> the assert-printing does not seem to show the value of that global >> variable? > > I noticed this as well and consider it a bug, yes. https://bitbucket.org/hpk42/pytest/issue/171/global-variables-are-not-explained-on >> > I also uploaded a new package pytest-2.3.0.dev8 to be installed >> > via: >> > >> > pip install -i http://pypi.testrun.org -U pytest >> >> I was playing with this over lunch and discovered this doesn't work: >> >> @pytest.factory(scope='session') >> def pytest_funcarg__foo(): >> return 42 >> >> Would it not make sense to allow this (or at least provide a clearer >> error)? I still like that form because of the grep-ability (doing a >> 2-line grep is much harder and would still not cover ppl doing "from >> pytest import factory" etc). > > Grepability is an argument. Would adding a "name=..." parameter for > the factory-decorator help enough? Allowing and advertising > pytest_funcarg__foo feels strange to somehow taking a fresh look i think. I would argue the opposite, allowing the @factory decroator on pytest_funcarg__* seems like a more gentle progression giving more the impression that it is simply funcargs evolved. To a newcomer it might otherwise look like funcargs where not thought out fully yet and make them think py.test just isn't stable enough yet. >> Also doing this results in setup_module being called twice: >> >> @pytest.setup(scope='module') >> def setup_module(): >> print 'setting up module' >> >> I'm not sure what the correct behaviour should be here. > > Hum, I think the decorator "consumes" the function and it should not > be considered for anything else. Do you agree? Yes, that's probably the best option. Regards, Floris From adam at goucher.ca Thu Aug 2 20:49:19 2012 From: adam at goucher.ca (Adam Goucher) Date: Thu, 02 Aug 2012 14:49:19 -0400 Subject: [py-dev] xdist and pytest.main Message-ID: I have a WebDriver framework that wraps Py.Test and after a bunch of setup stuff calls into things with run_status = pytest.main(args=arguments, plugins=[marks.MarksDecorator()]) which works fine for a single execution. But now I've got a couple clients that /really/ want parallel execution. My thinking for this would be to add in the necessary arguments to pass in the way all the other arguments have been... if 'n' in results.__dict__ and results.__dict__['n'] != None: arguments.append("--dist=load") arguments.append("--tx=%s*popen" % results.__dict__['n']) But my scripts are not actually executing. (Which shouldnt be too too surprising since I likely wouldnt be writing an email if they were...) Adam-Gouchers-MacBook:ebay adam$ pysaunter.py -m shirts -n 2 ==================================================================== test session starts ===================================================================== platform darwin -- Python 2.7.2 -- pytest-2.2.4 gw0 [0] / gw1 [0] scheduling tests via LoadScheduling ----------------------------- generated xml file: /Users/adam/work/saunter/py.saunter-examples/ebay/logs/2012-08-02-14-21-46.xml ----------------------------- ====================================================================== in 1.21 seconds ====================================================================== Adam-Gouchers-MacBook:ebay adam$ Any suggestions on how to debug why the workers are being created but the scripts not executed? I'd like to not write my own xdist style plugin so would like to make things behave with the existing one. -adam From holger at merlinux.eu Thu Aug 2 22:03:03 2012 From: holger at merlinux.eu (holger krekel) Date: Thu, 2 Aug 2012 20:03:03 +0000 Subject: [py-dev] new resource API nearing completion including impl In-Reply-To: References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> <20120802172419.GF26019@merlinux.eu> Message-ID: <20120802200303.GG26019@merlinux.eu> On Thu, Aug 02, 2012 at 19:47 +0100, Floris Bruynooghe wrote: > On 2 August 2012 18:24, holger krekel wrote: > > On Thu, Aug 02, 2012 at 13:50 +0100, Floris Bruynooghe wrote: > >> On 2 August 2012 11:44, holger krekel wrote: > >> > http://pytest.org/dev/setup.html > >> > > >> > Hope the latter begins to make more sense. > >> > >> Yes, it does. I now see the power @setup. One thing you might want > >> to add is compare the module-global setting to simply using the > >> "global" statement inside the setup function. > > > > Do you mean that in the case where the global-setting happens in the > > conftest.py using "global" does not work? > > They both work, I meant: > > @pytest.setup(scope='module') > def mysetup(): > global myvariable > myvariable = 42 > > vs: > > @pytest.setup(scope='module') > def mysetup(testcontext): > testcontext.module.myvariable = 42 > > Intuitively I would go for the first option and I think most python > developers would as well since it is what they know. I know the > second option, which you demonstrate, is possible from inside a plugin > or a conftest.py but I probably would consider this a bad practice > since they are not logical places to inject globals. Actually > injecting globals seems weird to me, I used to do this when having > plain old unittest but ever since funcargs I just type in the argument > and find that much clearer. So in short, while it is nice you can do > this I don't think it is a great example as, IMHO, it encourages a bad > practice. I was guided by having seen questions which ask for things like this. But it's not good as a primary example, i agree. > Would a better example be checking for a mark on the function and > doing some special setup based on that? Maybe the special setup is > requesting another funcarg/resource? (But there doesn't seem to be an > API on the testcontext to request a custom funcarg/resource. Is this > meant to be disallowed?) setup based on markers is a good example, yes, not sure it's good as a first basic example, though, because it introduces several examples at once. And yes, testcontext does not provide getfuncargvalue()/cached_setup on purpose. It's breaks parametrization because factory-func bodies are opaque so pytest cannot know that you are using a parametrized resource. I basically aim to design the new API to be totally safe. Other than that, testcontext can grow functionality ... > Which brings me onto the next topic, the testcontext object provides > access to .function, which you can use to check marks etc. But when > you had merged items/requests we had access to the item and you where > even talking about having item.applymarker() and a specific API for > inspecting markers. Would it not be nice to have these APIs on > testcontext (if the sope is "function")? ... Yes ... Maybe a "testcontext.getmarker(name)" returning None if it doesn't exist. And a testcontext.applymarker() as with the request object. The latter is maybe a bit harder to implement because testcontext has a scope whereas "request" was always function-scoped. And e.g. a @setup(scope=module) function calling testcontext.applymarker(pytest.mark.xfail) would probably expect to have it applied to all of its tests. Of course, we could restrict applymarker to "function" scope initially. > >> Btw, is it a bug in the assertion that when using a global variable > >> the assert-printing does not seem to show the value of that global > >> variable? > > > > I noticed this as well and consider it a bug, yes. > > https://bitbucket.org/hpk42/pytest/issue/171/global-variables-are-not-explained-on Oh, bitbucket has a new interface, nice (i think)! > >> > I also uploaded a new package pytest-2.3.0.dev8 to be installed > >> > via: > >> > > >> > pip install -i http://pypi.testrun.org -U pytest > >> > >> I was playing with this over lunch and discovered this doesn't work: > >> > >> @pytest.factory(scope='session') > >> def pytest_funcarg__foo(): > >> return 42 > >> > >> Would it not make sense to allow this (or at least provide a clearer > >> error)? I still like that form because of the grep-ability (doing a > >> 2-line grep is much harder and would still not cover ppl doing "from > >> pytest import factory" etc). > > > > Grepability is an argument. Would adding a "name=..." parameter for > > the factory-decorator help enough? Allowing and advertising > > pytest_funcarg__foo feels strange to somehow taking a fresh look i think. > > I would argue the opposite, allowing the @factory decroator on > pytest_funcarg__* seems like a more gentle progression giving more the > impression that it is simply funcargs evolved. To a newcomer it might > otherwise look like funcargs where not thought out fully yet and make > them think py.test just isn't stable enough yet. But when using the factory decorator on pytest_funcarg__ named functions, they shall at least not be able to receive "request" anymore, right? (The current implementation probably allows it but i feel uneasy about it). > >> Also doing this results in setup_module being called twice: > >> > >> @pytest.setup(scope='module') > >> def setup_module(): > >> print 'setting up module' > >> > >> I'm not sure what the correct behaviour should be here. > > > > Hum, I think the decorator "consumes" the function and it should not > > be considered for anything else. Do you agree? > > Yes, that's probably the best option. Can you open an issue about it as well? I don't think i get too much implementation time before going on vacation ... holger From holger at merlinux.eu Thu Aug 2 22:12:48 2012 From: holger at merlinux.eu (holger krekel) Date: Thu, 2 Aug 2012 20:12:48 +0000 Subject: [py-dev] xdist and pytest.main In-Reply-To: References: Message-ID: <20120802201248.GH26019@merlinux.eu> Hi Adam, On Thu, Aug 02, 2012 at 14:49 -0400, Adam Goucher wrote: > I have a WebDriver framework that wraps Py.Test and after a bunch of > setup stuff calls into things with > > run_status = pytest.main(args=arguments, plugins=[marks.MarksDecorator()]) > > which works fine for a single execution. But now I've got a couple > clients that /really/ want parallel execution. My thinking for this > would be to add in the necessary arguments to pass in the way all the > other arguments have been... > > if 'n' in results.__dict__ and results.__dict__['n'] != None: > arguments.append("--dist=load") > arguments.append("--tx=%s*popen" % results.__dict__['n']) > > But my scripts are not actually executing. (Which shouldnt be too too > surprising since I likely wouldnt be writing an email if they were...) > > Adam-Gouchers-MacBook:ebay adam$ pysaunter.py -m shirts -n 2 > ==================================================================== > test session starts > ===================================================================== > platform darwin -- Python 2.7.2 -- pytest-2.2.4 > gw0 [0] / gw1 [0] > scheduling tests via LoadScheduling > > ----------------------------- generated xml file: > /Users/adam/work/saunter/py.saunter-examples/ebay/logs/2012-08-02-14-21-46.xml > ----------------------------- > ====================================================================== > in 1.21 seconds > ====================================================================== > Adam-Gouchers-MacBook:ebay adam$ > > Any suggestions on how to debug why the workers are being created but > the scripts not executed? I'd like to not write my own xdist style > plugin so would like to make things behave with the existing one. Hum, this looks like no tests are collected at all. If you leave away the "-n" option, tests do run? Can you show the -v output of that? I assume you are running things in the correct directory and have no change-directory code in your tests/plugin? best, holger > -adam > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From holger at merlinux.eu Fri Aug 3 09:28:46 2012 From: holger at merlinux.eu (holger krekel) Date: Fri, 3 Aug 2012 07:28:46 +0000 Subject: [py-dev] naming issues (was: Re: new resource API nearing completion) In-Reply-To: References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> <20120802172419.GF26019@merlinux.eu> Message-ID: <20120803072845.GI26019@merlinux.eu> On Thu, Aug 02, 2012 at 19:47 +0100, Floris Bruynooghe wrote: > On 2 August 2012 18:24, holger krekel wrote: > > On Thu, Aug 02, 2012 at 13:50 +0100, Floris Bruynooghe wrote: > >> @pytest.factory(scope='session') > >> def pytest_funcarg__foo(): > >> return 42 > >> > >> Would it not make sense to allow this (or at least provide a clearer > >> error)? I still like that form because of the grep-ability (doing a > >> 2-line grep is much harder and would still not cover ppl doing "from > >> pytest import factory" etc). > > > > Grepability is an argument. Would adding a "name=..." parameter for > > the factory-decorator help enough? Allowing and advertising > > pytest_funcarg__foo feels strange to somehow taking a fresh look i think. > > I would argue the opposite, allowing the @factory decroator on > pytest_funcarg__* seems like a more gentle progression giving more the > impression that it is simply funcargs evolved. To a newcomer it might > otherwise look like funcargs where not thought out fully yet and make > them think py.test just isn't stable enough yet. If we are to optionally allow pytest_funcarg__ naming with @pytest.factory i wonder if renaming @pytest.factory to @pytest.funcarg would make sense and contribute to the notion that pytest-2.3 just extends funcarg facilities. The reason i favored @factory is that it really marks a factory function whereas "funcarg" more denotes the argument in a test or setup function. Moreover, the pytest-2.2 documentation already talked about factories when refering to pytest_funcarg__ functions. (And @funcargfactory seems a tad long to me). However, i can't claim to have much of an outside view so i am interested in your and other opinions. best, holger > > >> Also doing this results in setup_module being called twice: > >> > >> @pytest.setup(scope='module') > >> def setup_module(): > >> print 'setting up module' > >> > >> I'm not sure what the correct behaviour should be here. > > > > Hum, I think the decorator "consumes" the function and it should not > > be considered for anything else. Do you agree? > > Yes, that's probably the best option. > > > Regards, > Floris > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From ralf at systemexit.de Fri Aug 3 09:53:37 2012 From: ralf at systemexit.de (Ralf Schmitt) Date: Fri, 03 Aug 2012 09:53:37 +0200 Subject: [py-dev] naming issues In-Reply-To: <20120803072845.GI26019@merlinux.eu> (holger krekel's message of "Fri, 3 Aug 2012 07:28:46 +0000") References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> <20120802172419.GF26019@merlinux.eu> <20120803072845.GI26019@merlinux.eu> Message-ID: <877gtg8l8e.fsf@winserver.brainbot.com> holger krekel writes: > > If we are to optionally allow pytest_funcarg__ naming with @pytest.factory > i wonder if renaming @pytest.factory to @pytest.funcarg would make sense > and contribute to the notion that pytest-2.3 just extends funcarg > facilities. The reason i favored @factory is that it really marks a > factory function whereas "funcarg" more denotes the argument in a test > or setup function. Moreover, the pytest-2.2 documentation already talked > about factories when refering to pytest_funcarg__ functions. > (And @funcargfactory seems a tad long to me). > > However, i can't claim to have much of an outside view so i am interested > in your and other opinions. +1 for @pytest.funcarg (mostly because I like to grep for it) From holger at merlinux.eu Fri Aug 3 10:51:23 2012 From: holger at merlinux.eu (holger krekel) Date: Fri, 3 Aug 2012 08:51:23 +0000 Subject: [py-dev] naming issues In-Reply-To: <877gtg8l8e.fsf@winserver.brainbot.com> References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> <20120802172419.GF26019@merlinux.eu> <20120803072845.GI26019@merlinux.eu> <877gtg8l8e.fsf@winserver.brainbot.com> Message-ID: <20120803085123.GJ26019@merlinux.eu> On Fri, Aug 03, 2012 at 09:53 +0200, Ralf Schmitt wrote: > holger krekel writes: > > > > > If we are to optionally allow pytest_funcarg__ naming with @pytest.factory > > i wonder if renaming @pytest.factory to @pytest.funcarg would make sense > > and contribute to the notion that pytest-2.3 just extends funcarg > > facilities. The reason i favored @factory is that it really marks a > > factory function whereas "funcarg" more denotes the argument in a test > > or setup function. Moreover, the pytest-2.2 documentation already talked > > about factories when refering to pytest_funcarg__ functions. > > (And @funcargfactory seems a tad long to me). > > > > However, i can't claim to have much of an outside view so i am interested > > in your and other opinions. > > +1 for @pytest.funcarg (mostly because I like to grep for it) But that would work with "grep @pytest.factory" as well, wouldn't it? From ralf at systemexit.de Fri Aug 3 11:15:33 2012 From: ralf at systemexit.de (Ralf Schmitt) Date: Fri, 03 Aug 2012 11:15:33 +0200 Subject: [py-dev] naming issues In-Reply-To: <20120803085123.GJ26019@merlinux.eu> (holger krekel's message of "Fri, 3 Aug 2012 08:51:23 +0000") References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> <20120802172419.GF26019@merlinux.eu> <20120803072845.GI26019@merlinux.eu> <877gtg8l8e.fsf@winserver.brainbot.com> <20120803085123.GJ26019@merlinux.eu> Message-ID: <87pq7872ve.fsf@winserver.brainbot.com> holger krekel writes: > On Fri, Aug 03, 2012 at 09:53 +0200, Ralf Schmitt wrote: >> holger krekel writes: >> >> > >> > If we are to optionally allow pytest_funcarg__ naming with @pytest.factory >> > i wonder if renaming @pytest.factory to @pytest.funcarg would make sense >> > and contribute to the notion that pytest-2.3 just extends funcarg >> > facilities. The reason i favored @factory is that it really marks a >> > factory function whereas "funcarg" more denotes the argument in a test >> > or setup function. Moreover, the pytest-2.2 documentation already talked >> > about factories when refering to pytest_funcarg__ functions. >> > (And @funcargfactory seems a tad long to me). >> > >> > However, i can't claim to have much of an outside view so i am interested >> > in your and other opinions. >> >> +1 for @pytest.funcarg (mostly because I like to grep for it) > > But that would work with "grep @pytest.factory" as well, wouldn't it? probably, but that doesn't give me old funcarg factory functions. I have to admit that I didn't read the proposals and didn't follow the discussion closely. Calling the thing factory might be too broad. If pytest.funcarg does make sense (and you said it does), I'm now +2 for it. And since it looks like this should just be a shorter name for funcargfactory, I think 'funcarg' carries more meaning. With my previous vote that makes it +3 :) -- Cheers Ralf From Ronny.Pfannschmidt at gmx.de Fri Aug 3 11:33:33 2012 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Fri, 03 Aug 2012 11:33:33 +0200 Subject: [py-dev] naming issues In-Reply-To: <87pq7872ve.fsf@winserver.brainbot.com> References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> <20120802172419.GF26019@merlinux.eu> <20120803072845.GI26019@merlinux.eu> <877gtg8l8e.fsf@winserver.brainbot.com> <20120803085123.GJ26019@merlinux.eu> <87pq7872ve.fsf@winserver.brainbot.com> Message-ID: <501B9AED.10707@gmx.de> I dont like the name funcarg for those cases, since its not really descriptive of what it is anymore Naming things different from what they are creates extra mental ballast i don't want to have back when we only had funcargs, the name "funcarg" was descriptive, now it isn't anymore, since we have a lot more On 08/03/2012 11:15 AM, Ralf Schmitt wrote: > holger krekel writes: > >> On Fri, Aug 03, 2012 at 09:53 +0200, Ralf Schmitt wrote: >>> holger krekel writes: >>> >>>> >>>> If we are to optionally allow pytest_funcarg__ naming with @pytest.factory >>>> i wonder if renaming @pytest.factory to @pytest.funcarg would make sense >>>> and contribute to the notion that pytest-2.3 just extends funcarg >>>> facilities. The reason i favored @factory is that it really marks a >>>> factory function whereas "funcarg" more denotes the argument in a test >>>> or setup function. Moreover, the pytest-2.2 documentation already talked >>>> about factories when refering to pytest_funcarg__ functions. >>>> (And @funcargfactory seems a tad long to me). >>>> >>>> However, i can't claim to have much of an outside view so i am interested >>>> in your and other opinions. >>> >>> +1 for @pytest.funcarg (mostly because I like to grep for it) >> >> But that would work with "grep @pytest.factory" as well, wouldn't it? > > probably, but that doesn't give me old funcarg factory functions. > > I have to admit that I didn't read the proposals and didn't follow the > discussion closely. Calling the thing factory might be too broad. If > pytest.funcarg does make sense (and you said it does), I'm now +2 for > it. > > And since it looks like this should just be a shorter name for > funcargfactory, I think 'funcarg' carries more meaning. > > With my previous vote that makes it +3 :) > From flub at devork.be Fri Aug 3 11:43:43 2012 From: flub at devork.be (Floris Bruynooghe) Date: Fri, 3 Aug 2012 10:43:43 +0100 Subject: [py-dev] naming issues (was: Re: new resource API nearing completion) In-Reply-To: <20120803072845.GI26019@merlinux.eu> References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> <20120802172419.GF26019@merlinux.eu> <20120803072845.GI26019@merlinux.eu> Message-ID: I'm not sure I can come up with an argument for either. @factory seems fine if it's already referred as funcarg factory. Essentially you could say the decorator marks it as a factory, which happens to be implemented using a funcarg I guess. I think I've just argued for @factory :-) On Aug 3, 2012 8:28 AM, "holger krekel" wrote: > On Thu, Aug 02, 2012 at 19:47 +0100, Floris Bruynooghe wrote: > > On 2 August 2012 18:24, holger krekel wrote: > > > On Thu, Aug 02, 2012 at 13:50 +0100, Floris Bruynooghe wrote: > > >> @pytest.factory(scope='session') > > >> def pytest_funcarg__foo(): > > >> return 42 > > >> > > >> Would it not make sense to allow this (or at least provide a clearer > > >> error)? I still like that form because of the grep-ability (doing a > > >> 2-line grep is much harder and would still not cover ppl doing "from > > >> pytest import factory" etc). > > > > > > Grepability is an argument. Would adding a "name=..." parameter for > > > the factory-decorator help enough? Allowing and advertising > > > pytest_funcarg__foo feels strange to somehow taking a fresh look i > think. > > > > I would argue the opposite, allowing the @factory decroator on > > pytest_funcarg__* seems like a more gentle progression giving more the > > impression that it is simply funcargs evolved. To a newcomer it might > > otherwise look like funcargs where not thought out fully yet and make > > them think py.test just isn't stable enough yet. > > If we are to optionally allow pytest_funcarg__ naming with @pytest.factory > i wonder if renaming @pytest.factory to @pytest.funcarg would make sense > and contribute to the notion that pytest-2.3 just extends funcarg > facilities. The reason i favored @factory is that it really marks a > factory function whereas "funcarg" more denotes the argument in a test > or setup function. Moreover, the pytest-2.2 documentation already talked > about factories when refering to pytest_funcarg__ functions. > (And @funcargfactory seems a tad long to me). > > However, i can't claim to have much of an outside view so i am interested > in your and other opinions. > > best, > holger > > > > > > > > > >> Also doing this results in setup_module being called twice: > > >> > > >> @pytest.setup(scope='module') > > >> def setup_module(): > > >> print 'setting up module' > > >> > > >> I'm not sure what the correct behaviour should be here. > > > > > > Hum, I think the decorator "consumes" the function and it should not > > > be considered for anything else. Do you agree? > > > > Yes, that's probably the best option. > > > > > > Regards, > > Floris > > _______________________________________________ > > py-dev mailing list > > py-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/py-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf at systemexit.de Fri Aug 3 12:01:48 2012 From: ralf at systemexit.de (Ralf Schmitt) Date: Fri, 03 Aug 2012 12:01:48 +0200 Subject: [py-dev] naming issues In-Reply-To: <501B9AED.10707@gmx.de> (Ronny Pfannschmidt's message of "Fri, 03 Aug 2012 11:33:33 +0200") References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> <20120802172419.GF26019@merlinux.eu> <20120803072845.GI26019@merlinux.eu> <877gtg8l8e.fsf@winserver.brainbot.com> <20120803085123.GJ26019@merlinux.eu> <87pq7872ve.fsf@winserver.brainbot.com> <501B9AED.10707@gmx.de> Message-ID: <87ehno70qb.fsf@winserver.brainbot.com> Ronny Pfannschmidt writes: > I dont like the name funcarg for those cases, > since its not really descriptive of what it is anymore > > Naming things different from what they are > creates extra mental ballast i don't want to have > > back when we only had funcargs, the name "funcarg" was descriptive, > now it isn't anymore, since we have a lot more this is the thing I don't know if the decorator is just about funcargs or not. so, please bear with me. anyway, I think factory isn't very descriptive either. I don't know what's the purpose of that decorator at the moment and the name "factory" basically gives me near zero information. -- Cheers Ralf From holger at merlinux.eu Fri Aug 3 18:37:07 2012 From: holger at merlinux.eu (holger krekel) Date: Fri, 3 Aug 2012 16:37:07 +0000 Subject: [py-dev] greppability of factories (was: Re: new resource API nearing completion including impl) In-Reply-To: <20120802200303.GG26019@merlinux.eu> References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> <20120802172419.GF26019@merlinux.eu> <20120802200303.GG26019@merlinux.eu> Message-ID: <20120803163707.GK26019@merlinux.eu> On Thu, Aug 02, 2012 at 20:03 +0000, holger krekel wrote: > On Thu, Aug 02, 2012 at 19:47 +0100, Floris Bruynooghe wrote: > > On 2 August 2012 18:24, holger krekel wrote: > > > On Thu, Aug 02, 2012 at 13:50 +0100, Floris Bruynooghe wrote: > > >> Would it not make sense to allow this (or at least provide a clearer > > >> error)? I still like that form because of the grep-ability (doing a > > >> 2-line grep is much harder and would still not cover ppl doing "from > > >> pytest import factory" etc). > > > > > > Grepability is an argument. Would adding a "name=..." parameter for > > > the factory-decorator help enough? Allowing and advertising > > > pytest_funcarg__foo feels strange to somehow taking a fresh look i think. > > > > I would argue the opposite, allowing the @factory decroator on > > pytest_funcarg__* seems like a more gentle progression giving more the > > impression that it is simply funcargs evolved. To a newcomer it might > > otherwise look like funcargs where not thought out fully yet and make > > them think py.test just isn't stable enough yet. > > But when using the factory decorator on pytest_funcarg__ named functions, > they shall at least not be able to receive "request" anymore, right? > (The current implementation probably allows it but i feel uneasy about it). One more thought: What will actually happen if you grep for "def FUNCARGNAME" - does this not usually yield the location of your factory and very few or no false positives? holger From flub at devork.be Fri Aug 3 20:39:17 2012 From: flub at devork.be (Floris Bruynooghe) Date: Fri, 3 Aug 2012 20:39:17 +0200 Subject: [py-dev] greppability of factories (was: Re: new resource API nearing completion including impl) In-Reply-To: <20120803163707.GK26019@merlinux.eu> References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> <20120802104453.GC26019@merlinux.eu> <20120802172419.GF26019@merlinux.eu> <20120802200303.GG26019@merlinux.eu> <20120803163707.GK26019@merlinux.eu> Message-ID: On 3 August 2012 18:37, holger krekel wrote: > On Thu, Aug 02, 2012 at 20:03 +0000, holger krekel wrote: >> On Thu, Aug 02, 2012 at 19:47 +0100, Floris Bruynooghe wrote: >> > On 2 August 2012 18:24, holger krekel wrote: >> > > On Thu, Aug 02, 2012 at 13:50 +0100, Floris Bruynooghe wrote: >> > >> Would it not make sense to allow this (or at least provide a clearer >> > >> error)? I still like that form because of the grep-ability (doing a >> > >> 2-line grep is much harder and would still not cover ppl doing "from >> > >> pytest import factory" etc). >> > > >> > > Grepability is an argument. Would adding a "name=..." parameter for >> > > the factory-decorator help enough? Allowing and advertising >> > > pytest_funcarg__foo feels strange to somehow taking a fresh look i think. >> > >> > I would argue the opposite, allowing the @factory decroator on >> > pytest_funcarg__* seems like a more gentle progression giving more the >> > impression that it is simply funcargs evolved. To a newcomer it might >> > otherwise look like funcargs where not thought out fully yet and make >> > them think py.test just isn't stable enough yet. >> >> But when using the factory decorator on pytest_funcarg__ named functions, >> they shall at least not be able to receive "request" anymore, right? >> (The current implementation probably allows it but i feel uneasy about it). Yes, I think the same logic applies as with decorating setup_module with @setup: it "consumes" the function and it's old meaning is gone. The only difference here is probably that it strips pytest_funcarg__ from the name. > One more thought: What will actually happen if you grep for "def > FUNCARGNAME" - does this not usually yield the location of your factory > and very few or no false positives? Yes, and easily spotted when using grep -B1. To be fair I don't think this is a major issue. We will still find our funcargs fine just as we find any other code. The old way was just super convenient and I think that is the only reason we're making a fuss. Regards, Floris From holger at merlinux.eu Sat Aug 4 11:32:38 2012 From: holger at merlinux.eu (holger krekel) Date: Sat, 4 Aug 2012 09:32:38 +0000 Subject: [py-dev] pytest FAQs on stackoverflow Message-ID: <20120804093238.GO26019@merlinux.eu> Hi everybody, i am subscribed to testing questions regarding pytest on http://stackoverflow.com/questions/tagged/py.test and i increasingly consider it the main FAQ system. Maybe some of you also want to subscribe (just hoover over the tag and hit "subscribe" if you have an SO account) to see latest questions and answers (or give answers). For the future, i'd like to see a selection of SO-FAQs on py.test integrated on the pytest web page itself but have no clue yet how that can be done. Does anybody here know? best, holger From holger at merlinux.eu Sat Aug 4 14:13:56 2012 From: holger at merlinux.eu (holger krekel) Date: Sat, 4 Aug 2012 12:13:56 +0000 Subject: [py-dev] optionally considering setup (needsdb usecase) In-Reply-To: <20120630112605.GA32749@laurie.devork.be> References: <20120629105523.GW11942@merlinux.eu> <20120630002304.GB26143@laurie.devork.be> <20120630080839.GY11942@merlinux.eu> <20120630112605.GA32749@laurie.devork.be> Message-ID: <20120804121356.GP26019@merlinux.eu> Hi Floris, all, i think the below from an earlier thread is an outstanding issue for the new API ... On Sat, Jun 30, 2012 at 12:26 +0100, Floris Bruynooghe wrote: > As an aside however, one of my usecases for merged request/item > objects was so I could put setup in a session-wide scoped funcarg but > also automatically request this funcarg based on a mark:: > > def pytest_runtest_setup(item): > if 'needsdb' in item.keywords: # or a more explicit API > item.getresource('db') > > I understand that this will still be possible via:: > > def pytest_runtest_setup(item): > if 'needsdb' in item.keywords: > item.session.getresource('db') > > Or something similar to that. With the current @setup API this is not possible but it should be. I'd like to understand the exact use case a bit. What do you do with the "db" object here? I guess you cause side effects because you would otherwise just request a funcarg in the tests, right? If so, then i can imagine the following solution: @pytest.setup(enabled=myhelper) def perform_side_effect_with(db): ... The "enabled" helper would be called during collection so that pytest gets to know which tests will actually execute the setup function and its (potentially parametrized) required resources. It could look like this:: def myhelper(collectioncontext): return "needsdb" in collectioncontext.markers: and collectioncontext also carries module/class/function (depending on the scope specified with the setup). If the helper returns True then the setup is considered and thus receives the DB object. Do you think this would solve your use case? (Collectioncontext would not have a addfinalizer() and might in the future offer more collection-specific things). best, holger > > > Regards, > Floris > > -- > Debian GNU/Linux -- The Power of Freedom > www.debian.org | www.gnu.org | www.kernel.org > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From issues-reply at bitbucket.org Sun Aug 5 09:47:08 2012 From: issues-reply at bitbucket.org (Shay Weiss) Date: Sun, 05 Aug 2012 07:47:08 -0000 Subject: [py-dev] [hpk42/pytest] Configuration Files - Add an option to read them from command line parameter or an environment variable (issue #174) Message-ID: --- you can reply above this line --- New issue 174: Configuration Files - Add an option to read them from command line parameter or an environment variable https://bitbucket.org/hpk42/pytest/issue/174/configuration-files-add-an-option-to-read Shay Weiss: Use case: I would like to be able to run a simple "py.test" discovery from the root of the trunk and not have the configuration files clutter the root. I would expect to be able to either supply a different path to configuration files than that of the discovery via a command line argument: "py.test path/to/testdir -c path/to/configurationfile" Alternately, I would like to have an environment variable which will be checked prior to "path/to/testdir". -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From flub at devork.be Sun Aug 5 17:14:33 2012 From: flub at devork.be (Floris Bruynooghe) Date: Sun, 5 Aug 2012 17:14:33 +0200 Subject: [py-dev] optionally considering setup (needsdb usecase) In-Reply-To: <20120804121356.GP26019@merlinux.eu> References: <20120629105523.GW11942@merlinux.eu> <20120630002304.GB26143@laurie.devork.be> <20120630080839.GY11942@merlinux.eu> <20120630112605.GA32749@laurie.devork.be> <20120804121356.GP26019@merlinux.eu> Message-ID: Hello Holger, On 4 August 2012 14:13, holger krekel wrote: > On Sat, Jun 30, 2012 at 12:26 +0100, Floris Bruynooghe wrote: >> As an aside however, one of my usecases for merged request/item >> objects was so I could put setup in a session-wide scoped funcarg but >> also automatically request this funcarg based on a mark:: >> >> def pytest_runtest_setup(item): >> if 'needsdb' in item.keywords: # or a more explicit API >> item.getresource('db') >> >> I understand that this will still be possible via:: >> >> def pytest_runtest_setup(item): >> if 'needsdb' in item.keywords: >> item.session.getresource('db') >> >> Or something similar to that. > > With the current @setup API this is not possible but it should be. I'd like > to understand the exact use case a bit. What do you do with the "db" > object here? I guess you cause side effects because you would otherwise > just request a funcarg in the tests, right? Actually there is no side effect here. This was born out of Andreas Pelme's desire to be able to mark tests with a marker while trying to re-use the session-wide caching that funcargs gave us. But the new @setup already covers this case completely since it can handle the caching just fine. But I still think this is a nice use-case, since it would allow being able to use the same setup and request it with either a funcargs or a mark. > If so, then i can imagine the following solution: > > @pytest.setup(enabled=myhelper) > def perform_side_effect_with(db): > ... > > The "enabled" helper would be called during collection so that > pytest gets to know which tests will actually execute the setup > function and its (potentially parametrized) required resources. > It could look like this:: > > def myhelper(collectioncontext): > return "needsdb" in collectioncontext.markers: > > and collectioncontext also carries module/class/function (depending on > the scope specified with the setup). If the helper returns True then > the setup is considered and thus receives the DB object. Do you > think this would solve your use case? (Collectioncontext would not > have a addfinalizer() and might in the future offer more collection-specific > things). This does sound like a very neat solution indeed, I think this would be a good addition. Regards, Floris From adam at goucher.ca Sun Aug 5 23:31:33 2012 From: adam at goucher.ca (Adam Goucher) Date: Sun, 05 Aug 2012 17:31:33 -0400 Subject: [py-dev] xdist and pytest.main In-Reply-To: <501ED29E.9020005@goucher.ca> References: <501ACBAF.4030500@goucher.ca> <20120802201248.GH26019@merlinux.eu> <501ED29E.9020005@goucher.ca> Message-ID: Whoops. Didn't look at the list reply-to settings so pulling the list back in. This does seem to be the cause. I commented out the sys.path.append in the wrapper and added def pytest_configure(config): sys.path.append(os.path.join(os.getcwd(), "modules")) to the conftest.py in the root and it 'seems' to be working. My initial thoughts around error detection still stands though. Thanks for the nudge to finding the solution. -adam > A little more debugging (including the --debug flag) has led me to > something I believe... > > For context my wrapper runs from a prescribed directory structure > shown at > https://github.com/Element-34/Py.Saunter-Examples/tree/master/ebay > (for example). And part of that wrapper is a modification to the > system path as such > > sys.path.append(os.path.join(cwd, "modules")) > > What I am now guessing is that the environment does not get forked > into the slave processes as alluded to in this message. > > [slave-gw1] sending collectreport {'data': {'longrepr': > 'scripts/DressShirts.py:15: in \n> from > saunter.testcase.webdriver import > SaunterTestCase\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pytest-2.2.4-py2.7.egg/_pytest/assertion/rewrite.py:156: > in load_module\n> py.builtin.exec_(co, > mod.__dict__)\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py.saunter-0.48-py2.7.egg/saunter/testcase/webdriver.py:34: > in \n> from tailored.webdriver import WebDriver\nE > ImportError: No module named tailored.webdriver', 'outcome': 'failed', > 'sections': [], 'result': None, 'nodeid': 'scripts/DressShirts.py'}} > > If that is the case [and I have to go fix my sister's email else I > would debug it further] > > a) how do I pass that environment change to my forked processes > (guessing just move that to conftest.py?) > b) if there is an exception thrown in the collection process on a > slave, it likely should bubble up to the user > > -adam >> Hum, this looks like no tests are collected at all. If you leave >> away the "-n" option, tests do run? Can you show the -v output of >> that? I assume you are running things in the correct directory >> and have no change-directory code in your tests/plugin? >> >> best, >> holger >> > From carl at oddbird.net Mon Aug 6 05:47:08 2012 From: carl at oddbird.net (Carl Meyer) Date: Sun, 05 Aug 2012 21:47:08 -0600 Subject: [py-dev] new resource API nearing completion including impl In-Reply-To: <20120802073639.GZ26019@merlinux.eu> References: <20120801131802.GY26019@merlinux.eu> <20120802073639.GZ26019@merlinux.eu> Message-ID: <501F3E3C.7050403@oddbird.net> Hi all, On 08/02/2012 01:36 AM, holger krekel wrote: > As to Django, maybe Carl can help by stating his guess if it's possible at all > to successively instantiate Django with different backends within > one process. I've just returned from vacation and haven't fully followed the thread, but I'd expect this to be difficult and fragile; the Django ORM currently instantiates a number of objects based on your settings (including the DATABASES setting) at import-time of the django.db module, so you'd either need to manually reinstantiate and monkey-patch those objects (e.g. django.db.connections), or reload django.db. It might also be possible to achieve this via Django's multiple-databases support somehow, though I'd expect again you'd need to do some monkey-patching of django.db.connections["default"], and there might well be hidden gotchas. Carl From issues-reply at bitbucket.org Mon Aug 6 07:05:18 2012 From: issues-reply at bitbucket.org (Anonymous) Date: Mon, 06 Aug 2012 05:05:18 -0000 Subject: [py-dev] [hpk42/pytest] way to control how pytest-xdist runs tests in parallel? (issue #175) Message-ID: <41b7176235f7c64b50ddbe4512fe9d47@bitbucket.org> --- you can reply above this line --- New issue 175: way to control how pytest-xdist runs tests in parallel? https://bitbucket.org/hpk42/pytest/issue/175/way-to-control-how-pytest-xdist-runs-tests Anonymous: Apologies if this has been raised and triaged already. I could not find it in the archives. The enhancement I request in py.test is the ability to control how tests are executed in parallel. It would be _really_ nice to have a way to run multiple classes/modules in parallel, while having serial execution within a given set of classes/modules. TestNG (Java) provides similar functionality. Please see: http://testng.org/doc/documentation-main.html#parallel-running This was also discussed a while ago on StackOverflow. Please see discussion: http://stackoverflow.com/questions/4637036/is-there-a-way-to-control-how-pytest-xdist-runs-tests-in-parallel -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From holger at merlinux.eu Mon Aug 6 09:28:05 2012 From: holger at merlinux.eu (holger krekel) Date: Mon, 6 Aug 2012 07:28:05 +0000 Subject: [py-dev] [hpk42/pytest] Configuration Files - Add an option to read them from command line parameter or an environment variable (issue #174) In-Reply-To: References: Message-ID: <20120806072805.GQ26019@merlinux.eu> Hi Shay, i agree this would be a very useful addition. Holger P.S.: also testing if bitbucket's issue-reply address routes my message to the issue ... On Sun, Aug 05, 2012 at 07:47 -0000, Shay Weiss wrote: > --- you can reply above this line --- > > New issue 174: Configuration Files - Add an option to read them from command line parameter or an environment variable > https://bitbucket.org/hpk42/pytest/issue/174/configuration-files-add-an-option-to-read > > Shay Weiss: > > Use case: I would like to be able to run a simple "py.test" discovery from the root of the trunk and not have the configuration files clutter the root. > > I would expect to be able to either supply a different path to configuration files than that of the discovery via a command line argument: > "py.test path/to/testdir -c path/to/configurationfile" > > Alternately, I would like to have an environment variable which will be checked prior to "path/to/testdir". > > > > > > > > > > -- > > This is an issue notification from bitbucket.org. You are receiving > this either because you are the owner of the issue, or you are > following the issue. > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From holger at merlinux.eu Mon Aug 6 09:50:49 2012 From: holger at merlinux.eu (holger krekel) Date: Mon, 6 Aug 2012 07:50:49 +0000 Subject: [py-dev] optionally considering setup (needsdb usecase) In-Reply-To: References: <20120629105523.GW11942@merlinux.eu> <20120630002304.GB26143@laurie.devork.be> <20120630080839.GY11942@merlinux.eu> <20120630112605.GA32749@laurie.devork.be> <20120804121356.GP26019@merlinux.eu> Message-ID: <20120806075049.GT26019@merlinux.eu> Hi Floris, On Sun, Aug 05, 2012 at 17:14 +0200, Floris Bruynooghe wrote: > Hello Holger, > > On 4 August 2012 14:13, holger krekel wrote: > > On Sat, Jun 30, 2012 at 12:26 +0100, Floris Bruynooghe wrote: > >> As an aside however, one of my usecases for merged request/item > >> objects was so I could put setup in a session-wide scoped funcarg but > >> also automatically request this funcarg based on a mark:: > >> > >> def pytest_runtest_setup(item): > >> if 'needsdb' in item.keywords: # or a more explicit API > >> item.getresource('db') > >> > >> I understand that this will still be possible via:: > >> > >> def pytest_runtest_setup(item): > >> if 'needsdb' in item.keywords: > >> item.session.getresource('db') > >> > >> Or something similar to that. > > > > With the current @setup API this is not possible but it should be. I'd like > > to understand the exact use case a bit. What do you do with the "db" > > object here? I guess you cause side effects because you would otherwise > > just request a funcarg in the tests, right? > > Actually there is no side effect here. This was born out of Andreas > Pelme's desire to be able to mark tests with a marker while trying to > re-use the session-wide caching that funcargs gave us. But the new > @setup already covers this case completely since it can handle the > caching just fine. But I still think this is a nice use-case, since > it would allow being able to use the same setup and request it with > either a funcargs or a mark. But if you request it with a mark, we are talking about side effects, aren't we? (I'd define "side-effects" as something that doesn't inject a test dependency directly to a test). > > If so, then i can imagine the following solution: > > > > @pytest.setup(enabled=myhelper) > > def perform_side_effect_with(db): > > ... > > > > The "enabled" helper would be called during collection so that > > pytest gets to know which tests will actually execute the setup > > function and its (potentially parametrized) required resources. > > It could look like this:: > > > > def myhelper(collectioncontext): > > return "needsdb" in collectioncontext.markers: > > > > and collectioncontext also carries module/class/function (depending on > > the scope specified with the setup). If the helper returns True then > > the setup is considered and thus receives the DB object. Do you > > think this would solve your use case? (Collectioncontext would not > > have a addfinalizer() and might in the future offer more collection-specific > > things). > > This does sound like a very neat solution indeed, I think this would > be a good addition. OK, i'll see to implement it but i guess it wouldn't be too bad to do it after a pytest-2.3 release, unless there is a concrete need already. Btw, if you can find some time sometime to look at a) test pytest-django with pytest-trunk b) port pytest-django to pytest-trunk features, that would be super-helpful. My personal target for a release is end august but not before some more real-world beta usages have happened. best, holger > > Regards, > Floris > From flub at devork.be Mon Aug 6 22:55:28 2012 From: flub at devork.be (Floris Bruynooghe) Date: Mon, 6 Aug 2012 21:55:28 +0100 Subject: [py-dev] optionally considering setup (needsdb usecase) In-Reply-To: <20120806075049.GT26019@merlinux.eu> References: <20120629105523.GW11942@merlinux.eu> <20120630002304.GB26143@laurie.devork.be> <20120630080839.GY11942@merlinux.eu> <20120630112605.GA32749@laurie.devork.be> <20120804121356.GP26019@merlinux.eu> <20120806075049.GT26019@merlinux.eu> Message-ID: On 6 August 2012 08:50, holger krekel wrote: > On Sun, Aug 05, 2012 at 17:14 +0200, Floris Bruynooghe wrote: >> On 4 August 2012 14:13, holger krekel wrote: >> > On Sat, Jun 30, 2012 at 12:26 +0100, Floris Bruynooghe wrote: >> >> As an aside however, one of my usecases for merged request/item >> >> objects was so I could put setup in a session-wide scoped funcarg but >> >> also automatically request this funcarg based on a mark:: >> >> >> >> def pytest_runtest_setup(item): >> >> if 'needsdb' in item.keywords: # or a more explicit API >> >> item.getresource('db') >> >> >> >> I understand that this will still be possible via:: >> >> >> >> def pytest_runtest_setup(item): >> >> if 'needsdb' in item.keywords: >> >> item.session.getresource('db') >> >> >> >> Or something similar to that. >> > >> > With the current @setup API this is not possible but it should be. I'd like >> > to understand the exact use case a bit. What do you do with the "db" >> > object here? I guess you cause side effects because you would otherwise >> > just request a funcarg in the tests, right? >> >> Actually there is no side effect here. This was born out of Andreas >> Pelme's desire to be able to mark tests with a marker while trying to >> re-use the session-wide caching that funcargs gave us. But the new >> @setup already covers this case completely since it can handle the >> caching just fine. But I still think this is a nice use-case, since >> it would allow being able to use the same setup and request it with >> either a funcargs or a mark. > > But if you request it with a mark, we are talking about side effects, aren't we? > (I'd define "side-effects" as something that doesn't inject a test > dependency directly to a test). Yes, I guess so. >> > If so, then i can imagine the following solution: >> > >> > @pytest.setup(enabled=myhelper) >> > def perform_side_effect_with(db): >> > ... >> > >> > The "enabled" helper would be called during collection so that >> > pytest gets to know which tests will actually execute the setup >> > function and its (potentially parametrized) required resources. >> > It could look like this:: >> > >> > def myhelper(collectioncontext): >> > return "needsdb" in collectioncontext.markers: >> > >> > and collectioncontext also carries module/class/function (depending on >> > the scope specified with the setup). If the helper returns True then >> > the setup is considered and thus receives the DB object. Do you >> > think this would solve your use case? (Collectioncontext would not >> > have a addfinalizer() and might in the future offer more collection-specific >> > things). >> >> This does sound like a very neat solution indeed, I think this would >> be a good addition. > > OK, i'll see to implement it but i guess it wouldn't be too bad to do > it after a pytest-2.3 release, unless there is a concrete need already. This would still make the dual setup triggered by mark/resource easier by e.g. (I'm just using some pseudo-api here): @pytest.funcarg def db(): print 'setting up db' def helper(collectioncontext): if 'needsdb' in collectioncontect.markers @pytest.setup(enabled=helper) def dbsetup(db): pass Here the funcarg can do all the work. But this is not a deal-breaker, since you can also do this: def setupdb(): print 'idempotent db setup' @pytest.funcarg(): def db(): setupdb() @pytest.setup def marksetup(testcontext): if 'needsdb' in testcontext.markers: setupdb() So leaving this till later is fine. > Btw, if you can find some time sometime to look at a) test > pytest-django with pytest-trunk b) port pytest-django to pytest-trunk > features, that would be super-helpful. My personal target for a release > is end august but not before some more real-world beta usages have happened. I'll try to have a go at this in the next week or two as I think it would be very good exercise as well. Regards, Floris From holger at merlinux.eu Tue Aug 7 22:16:30 2012 From: holger at merlinux.eu (holger krekel) Date: Tue, 7 Aug 2012 20:16:30 +0000 Subject: [py-dev] optionally considering setup (needsdb usecase) In-Reply-To: References: <20120629105523.GW11942@merlinux.eu> <20120630002304.GB26143@laurie.devork.be> <20120630080839.GY11942@merlinux.eu> <20120630112605.GA32749@laurie.devork.be> <20120804121356.GP26019@merlinux.eu> <20120806075049.GT26019@merlinux.eu> Message-ID: <20120807201630.GW26019@merlinux.eu> On Mon, Aug 06, 2012 at 21:55 +0100, Floris Bruynooghe wrote: > On 6 August 2012 08:50, holger krekel wrote: > > On Sun, Aug 05, 2012 at 17:14 +0200, Floris Bruynooghe wrote: > >> On 4 August 2012 14:13, holger krekel wrote: > >> > On Sat, Jun 30, 2012 at 12:26 +0100, Floris Bruynooghe wrote: > >> >> As an aside however, one of my usecases for merged request/item > >> >> objects was so I could put setup in a session-wide scoped funcarg but > >> >> also automatically request this funcarg based on a mark:: > >> >> > >> >> def pytest_runtest_setup(item): > >> >> if 'needsdb' in item.keywords: # or a more explicit API > >> >> item.getresource('db') > >> >> > >> >> I understand that this will still be possible via:: > >> >> > >> >> def pytest_runtest_setup(item): > >> >> if 'needsdb' in item.keywords: > >> >> item.session.getresource('db') > >> >> > >> >> Or something similar to that. > >> > > >> > With the current @setup API this is not possible but it should be. I'd like > >> > to understand the exact use case a bit. What do you do with the "db" > >> > object here? I guess you cause side effects because you would otherwise > >> > just request a funcarg in the tests, right? > >> > >> Actually there is no side effect here. This was born out of Andreas > >> Pelme's desire to be able to mark tests with a marker while trying to > >> re-use the session-wide caching that funcargs gave us. But the new > >> @setup already covers this case completely since it can handle the > >> caching just fine. But I still think this is a nice use-case, since > >> it would allow being able to use the same setup and request it with > >> either a funcargs or a mark. > > > > But if you request it with a mark, we are talking about side effects, aren't we? > > (I'd define "side-effects" as something that doesn't inject a test > > dependency directly to a test). > > Yes, I guess so. > > >> > If so, then i can imagine the following solution: > >> > > >> > @pytest.setup(enabled=myhelper) > >> > def perform_side_effect_with(db): > >> > ... > >> > > >> > The "enabled" helper would be called during collection so that > >> > pytest gets to know which tests will actually execute the setup > >> > function and its (potentially parametrized) required resources. > >> > It could look like this:: > >> > > >> > def myhelper(collectioncontext): > >> > return "needsdb" in collectioncontext.markers: > >> > > >> > and collectioncontext also carries module/class/function (depending on > >> > the scope specified with the setup). If the helper returns True then > >> > the setup is considered and thus receives the DB object. Do you > >> > think this would solve your use case? (Collectioncontext would not > >> > have a addfinalizer() and might in the future offer more collection-specific > >> > things). > >> > >> This does sound like a very neat solution indeed, I think this would > >> be a good addition. > > > > OK, i'll see to implement it but i guess it wouldn't be too bad to do > > it after a pytest-2.3 release, unless there is a concrete need already. > > This would still make the dual setup triggered by mark/resource easier > by e.g. (I'm just using some pseudo-api here): > > @pytest.funcarg > def db(): > print 'setting up db' > > def helper(collectioncontext): > if 'needsdb' in collectioncontect.markers > > @pytest.setup(enabled=helper) > def dbsetup(db): > pass > > Here the funcarg can do all the work. But this is not a deal-breaker, > since you can also do this: > > def setupdb(): > print 'idempotent db setup' > > @pytest.funcarg(): > def db(): > setupdb() > > @pytest.setup > def marksetup(testcontext): > if 'needsdb' in testcontext.markers: > setupdb() > > So leaving this till later is fine. Right, there is one difference between the two solutions however. The first works even if "db" is parametrized: The dbsetup function (and all of the tests using it) will be executed multiple times. With the second example dbsetup may only execute once for tests using "db" implicitely (through the setup). The basic rule is: if resources appear as funcargs in a function signature (test, setup or factory functions) then pytest can make parametrization work without further ado. Any getfuncargvalue()-like dynamic resource access can/would break it. This is the reason for the new testcontext/collectioncontext object and its more minimal API. > > Btw, if you can find some time sometime to look at a) test > > pytest-django with pytest-trunk b) port pytest-django to pytest-trunk > > features, that would be super-helpful. My personal target for a release > > is end august but not before some more real-world beta usages have happened. > > I'll try to have a go at this in the next week or two as I think it > would be very good exercise as well. Great, thanks. best, holger From issues-reply at bitbucket.org Wed Aug 8 08:02:14 2012 From: issues-reply at bitbucket.org (Jim Garrison) Date: Wed, 08 Aug 2012 06:02:14 -0000 Subject: [py-dev] [hpk42/pytest] pytest.raises(AssertionError) fails with cython modules (issue #176) Message-ID: <9e74dce79ebffc05e2be7fbd2644d60b@bitbucket.org> --- you can reply above this line --- New issue 176: pytest.raises(AssertionError) fails with cython modules https://bitbucket.org/hpk42/pytest/issue/176/pytestraises-assertionerror-fails-with Jim Garrison: Define the files python_assert.py and cython_assert.pyx to be identical, each containing a simple function that raises AssertionError: {{{ #!python def raise_assertionerror(): assert False }}} I would expect both of the following tests to succeed under pytest: {{{ #!python import pytest import pyximport pyximport.install() from python_assert import raise_assertionerror as python_assert from cython_assert import raise_assertionerror as cython_assert def test_assertion(): with pytest.raises(AssertionError): python_assert() def test_cython_assertion(): with pytest.raises(AssertionError): cython_assert() }}} However, the cython test fails. This seems to be a problem with pytest because the equivalent unittest succeeds. Further, the pytest test succeeds if we call pytest.raises(Exception) instead of pytest.raises(AssertionError). Any idea what is wrong? -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From holger at merlinux.eu Thu Aug 9 19:48:50 2012 From: holger at merlinux.eu (holger krekel) Date: Thu, 9 Aug 2012 17:48:50 +0000 Subject: [py-dev] xdist and pytest.main In-Reply-To: References: <501ACBAF.4030500@goucher.ca> <20120802201248.GH26019@merlinux.eu> <501ED29E.9020005@goucher.ca> Message-ID: <20120809174850.GE26019@merlinux.eu> On Sun, Aug 05, 2012 at 17:31 -0400, Adam Goucher wrote: > Whoops. Didn't look at the list reply-to settings so pulling the list > back in. This does seem to be the cause. I commented out the > sys.path.append in the wrapper and added > > def pytest_configure(config): > sys.path.append(os.path.join(os.getcwd(), "modules")) > > to the conftest.py in the root and it 'seems' to be working. > > My initial thoughts around error detection still stands though. Your passing of --debug is also what i would have tried. One can also enable EXECNET debugging, see http://codespeak.net/execnet/basics.html#debugging-execnet This gives a lot of low-level network messages but is sometimes helpful. best, holger > Thanks for the nudge to finding the solution. > > -adam > > A little more debugging (including the --debug flag) has led me to > > something I believe... > > > > For context my wrapper runs from a prescribed directory structure > > shown at > > https://github.com/Element-34/Py.Saunter-Examples/tree/master/ebay > > (for example). And part of that wrapper is a modification to the > > system path as such > > > > sys.path.append(os.path.join(cwd, "modules")) > > > > What I am now guessing is that the environment does not get forked > > into the slave processes as alluded to in this message. > > > > [slave-gw1] sending collectreport {'data': {'longrepr': > > 'scripts/DressShirts.py:15: in \n> from > > saunter.testcase.webdriver import > > SaunterTestCase\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pytest-2.2.4-py2.7.egg/_pytest/assertion/rewrite.py:156: > > in load_module\n> py.builtin.exec_(co, > > mod.__dict__)\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py.saunter-0.48-py2.7.egg/saunter/testcase/webdriver.py:34: > > in \n> from tailored.webdriver import WebDriver\nE > > ImportError: No module named tailored.webdriver', 'outcome': 'failed', > > 'sections': [], 'result': None, 'nodeid': 'scripts/DressShirts.py'}} > > > > If that is the case [and I have to go fix my sister's email else I > > would debug it further] > > > > a) how do I pass that environment change to my forked processes > > (guessing just move that to conftest.py?) > > b) if there is an exception thrown in the collection process on a > > slave, it likely should bubble up to the user > > > > -adam > >> Hum, this looks like no tests are collected at all. If you leave > >> away the "-n" option, tests do run? Can you show the -v output of > >> that? I assume you are running things in the correct directory > >> and have no change-directory code in your tests/plugin? > >> > >> best, > >> holger > >> > > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From issues-reply at bitbucket.org Tue Aug 14 17:10:45 2012 From: issues-reply at bitbucket.org (Anonymous) Date: Tue, 14 Aug 2012 15:10:45 -0000 Subject: [py-dev] [hpk42/pytest] Test functions with leading underscores ignored (issue #177) Message-ID: --- you can reply above this line --- New issue 177: Test functions with leading underscores ignored https://bitbucket.org/hpk42/pytest/issue/177/test-functions-with-leading-underscores Anonymous: Why does _pytest/python.py contain this code in PyCollectorMixin.collect()?: if name[0] != "_": and can the restriction be removed, as I would like to make my test functions "private" to the module. Thanks for being awesome. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From flub at devork.be Wed Aug 15 00:41:11 2012 From: flub at devork.be (Floris Bruynooghe) Date: Tue, 14 Aug 2012 23:41:11 +0100 Subject: [py-dev] New resource API feedback Message-ID: Hello Holger, I've started experimenting a bit more with the new resource api in pytest-django, I haven't got very far yet but do have already some feedback and questions. Firstly my main issue, I don't know how to inspect the marks on the function/item in a function-scoped setup. Looking at the code the only thing I could find was TestContext._resource.keywords and TestContext._resource.applymarker(). The latter which has an explicit comment saying it is unavailable on purpose. The former almost exposed as TestContext.keywords but commented out. So how do you use markers? This should probably be documented as well. Secondly the docs should probably show how to do teardown in an @setup function. I think it would be nice to show an example of scope and teardown before going into the global resource example. Related to this TestContext.addfinalizer() is not documented in the TestContext API docs. Probably because autodoc doesn't pick it up. Maybe simply merging TestContextRequest into TestContext is enough? TestContextSetup would not need any changes to keep it's behaviour in that case. Next something I have mentioned before, marking a pytest_funcarg__foo function with @factory seems to sill give an incomprehensible error. Personally I think it should be possible and "consume" the funcarg just like @setup consumes e.g. setup_module(), but if I'm alone in that a clearer error would be good improvement. Another thing which surprised me was that @pytest.setup() needs to be called in order to have any effect. Not calling the decorator will simply ignore the setup function, I expected it to treat it as a function-scoped setup. Regards, Floris From holger at merlinux.eu Wed Aug 15 09:54:22 2012 From: holger at merlinux.eu (holger krekel) Date: Wed, 15 Aug 2012 07:54:22 +0000 Subject: [py-dev] New resource API feedback In-Reply-To: References: Message-ID: <20120815075422.GK26019@merlinux.eu> Hi Floris, On Tue, Aug 14, 2012 at 23:41 +0100, Floris Bruynooghe wrote: > Hello Holger, > > I've started experimenting a bit more with the new resource api in > pytest-django, I haven't got very far yet but do have already some > feedback and questions. > > Firstly my main issue, I don't know how to inspect the marks on the > function/item in a function-scoped setup. Looking at the code the > only thing I could find was TestContext._resource.keywords and > TestContext._resource.applymarker(). The latter which has an explicit > comment saying it is unavailable on purpose. The former almost > exposed as TestContext.keywords but commented out. So how do you use > markers? This should probably be documented as well. Accessing and working with markers is missing - i only briefly touched it during the implementation of the new resource API. I intend to have testcontext grow a "markers" dictionary, mapping mark names to lists of MarkInfo objects. You can currently work-around/hack using testcontext._request._pyfuncitem.obj as a reference to the underlying test function. > Secondly the docs should probably show how to do teardown in an @setup > function. I think it would be nice to show an example of scope and > teardown before going into the global resource example. Related to > this TestContext.addfinalizer() is not documented in the TestContext > API docs. Probably because autodoc doesn't pick it up. Maybe simply > merging TestContextRequest into TestContext is enough? > TestContextSetup would not need any changes to keep it's behaviour in > that case. Makes all sense i think. > Next something I have mentioned before, marking a pytest_funcarg__foo > function with @factory seems to sill give an incomprehensible error. > Personally I think it should be possible and "consume" the funcarg > just like @setup consumes e.g. setup_module(), but if I'm alone in > that a clearer error would be good improvement. Agreed, i'll see to lopok into it. > Another thing which surprised me was that @pytest.setup() needs to be > called in order to have any effect. Not calling the decorator will > simply ignore the setup function, I expected it to treat it as a > function-scoped setup. I wasn't quite sure if to mimick the current pytest.mark behaviour of allowing usage with and without "()". While trying to write a docstring for it i thought it's maybe better to allow just one way. But there definitely should be some clear error best, holger > > Regards, > Floris > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From issues-reply at bitbucket.org Wed Aug 15 23:20:00 2012 From: issues-reply at bitbucket.org (Jason R. Coombs) Date: Wed, 15 Aug 2012 21:20:00 -0000 Subject: [py-dev] [hpk42/pytest] stdout and stderr double-encoded in junitxml (issue #178) Message-ID: <5eae8d7c903208f28031d4d30f9a28fc@bitbucket.org> --- you can reply above this line --- New issue 178: stdout and stderr double-encoded in junitxml https://bitbucket.org/hpk42/pytest/issue/178/stdout-and-stderr-double-encoded-in Jason R. Coombs: I'm finding that the junit XML generated by pytest 2.2.4 gets double-encoded output. For example, here's a snippet from one run: {{{ ... Error in parsing gryphon/models/qsl.py:63: QSLErrorMessage 1 {&apos;exception&apos;: [&apos;ParseError: Both inline and named res ponse lists were used for question lookahead\n&apos;], &apos;message&apos;: [&apos;ParseError: Both inline and named respo nse lists were used for question lookahead\n&apos;], &apos;status&apos;: 500, ... }}} The stderr stream for that looks something like: {{{ Message 1 {'exception': ['ParseError: Both inline and named response lists were used for question lookahead\n'], 'message': ['ParseError: Both inline and named response lists wer eused for the queston lookahead\n'], 'status': 500, }}} Notice that the tickmarks are being escaped for XML (to ') and then again the ampersand is getting escaped to &apos;. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From brianna.laugher at gmail.com Thu Aug 16 10:58:37 2012 From: brianna.laugher at gmail.com (Brianna Laugher) Date: Thu, 16 Aug 2012 18:58:37 +1000 Subject: [py-dev] new resource API documentation comments Message-ID: Hi, I just spent some time reading the dev docs so these comments are just based on the docs and not actually using the new API. In general it looks pretty sensible. - being able to have funcargs like funcargs directly is really nice, a lot more obvious than calling request.getfuncargvalue('foo') - why addfinalizer and not teardown? -although I don't really know what cached_setup did, the trinity of defining the scope, setup and teardown methods made sense to me. Now the scope is in a decorator, the setup is implicitly the entire thing that is happening and the teardown seems somewhat awkwardly tacked on. - none of the "addfinalizer" examples take an argument, how would you convert an old-style teardown method to that? e.g. we have a lot of funcargs which do things like return request.cached_setup(setup=setup, teardown=lambda obj: obj.close(), scope='function') - Sometimes things are referred to as "funcargs", sometimes they are referred to as "injected resources". Is there any difference here? The funcarg is the actual function and the injected resource is the instance in a specific test function? I suggest to use the term "funcarg" as much as possible as it is specific and a necessary concept for using pytest with any depth. Some of the following comments are fairly picky so feel free to ignore them. funcargs.txt line 118 - I think in this first incarnation of the smtp funcarg (factory? what to call it now?), it doesn't actually need to take a testcontext, right? line 527 "Parametrizing test functions" - may be worth having a simple example showing a combination using both (test data) parametrization and funcarg parametrization, to emphasise how they are differently useful. Using a database as an example of funcarg parametrization is good, maybe better than values like 1/2. I feel like parametrizing tests (test data) is probably the more common use case and it is a little buried amongst the heavy duty parametrized funcargs. line 598 "Basic ``pytest_generate_tests`` example" - I think this is not a very basic example! I think it is copied from parametrize.txt page, where it might make more sense. Here is what I would consider a basic example. # code def isSquare(n): n = n ** 0.5 return int(n) == n # test file def pytest_generate_tests(metafunc): squares = [1, 4, 9, 16, 25, 36, 49] for n in range(1, 50): expected = n in squares if metafunc.function.__name__ == 'test_isSquare': metafunc.addcall(id=n, funcargs=dict(n=n, expected=expected)) def test_isSquare(n, expected): assert isSquare(n) == expected Well, this is so trivial you might bundle it all into a single test, but it can be useful to have each case genuinely be a separate test. You could also shoe-horn this data into a parametrize decorator I suppose but it is nicer to have more space if your test data is more complicated, to explain it. I am starting to have a feeling that the way my project has been using generate_tests is not the way everyone else uses it. In our conftest.py one of the enterprising developers on the project (who got us all onto py.test initially) put this: def pytest_generate_tests(__multicall__, metafunc): """Supports parametrised tests using generate_ fns. Use multicall to call any other pytest_generate_tests hooks first. If the test_ fn has a generate_ fn then call it with the metafunc to let it parametrise the test. """ __multicall__.execute() name = metafunc.function.__name__.replace('test_', 'generate_') fn = getattr(metafunc.module, name, None) if fn: fn(metafunc) this means I can simplify my example above to: def generate_isSquare(metafunc): squares = [1, 4, 9, 16, 25, 36, 49] for n in range(1, 50): expected = n in squares metafunc.addcall(id=n, funcargs=dict(n=n, expected=expected)) maybe this is a useful pattern for others? or maybe we're doing it wrong, I dunno :) in funcarg_compare.txt line 53 - (in old-style) "4. there is no way how you can make use of funcarg factories in xUnit setup methods." Is there an example of this? I would like to see it! This sounds like a way for people with existing xUnit style test suites to more easily convert to funcarg style. Maybe with all the parametrized test examples it is a good idea to show the verbose output, just to be more explicit about what is being generated. As an aside, there is a comment in example/markers.txt line 144 that "You can use the ``-k`` command line option to only run tests with names matching the given argument". Whether it is an oversight in the functionality or the docs, I have always found that -k also matches against markers (py.test.mark decorators). (Currently using pytest 2.2.3) in setup.txt line 65 'old = tmpdir.chdir()' - it's not really relevant to the example but I don't know what this does, and I couldn't find out, so it's just a bit distracting. thanks as always for providing this great library :) cheers Brianna -- They've just been waiting in a mountain for the right moment: http://modernthings.org/ From brianna.laugher at gmail.com Fri Aug 17 09:32:54 2012 From: brianna.laugher at gmail.com (Brianna Laugher) Date: Fri, 17 Aug 2012 17:32:54 +1000 Subject: [py-dev] new resource API documentation comments In-Reply-To: References: Message-ID: Also, Would this be a roughly equivalent old-style to the smtp examples in http://pytest.org/dev/funcargs.html ? def pytest_funcarg__smtpMerLinux(request): smtp = smtplib.SMTP("merlinux.eu") def teardown(smtp): print ("finalizing %s" % smtp) smtp.close() return request.cached_setup(setup=lambda: smtp, teardown=teardown, scope='session') def pytest_funcarg__smtpMailPython(request): smtp = smtplib.SMTP("mail.python.org") def teardown(smtp): print ("finalizing %s" % smtp) smtp.close() return request.cached_setup(setup=lambda: smtp, teardown=teardown, scope='session') # test file def pytest_generate_tests(metafunc): merlinux = request.getfuncargvalue('smtpMerLinux') mailpython = request.getfuncargvalue('smtpMailPython') if'smtp' in metafunc.funcargnames: metafunc.addcall(id='merlinux.eu', param=merlinux) # ? would this work? seems magic metafunc.addcall(id='mail.python.org', param=mailpython) I feel like this makes it clear how much more powerful parametrizing funcargs themselves is. cheers Brianna On 16 August 2012 18:58, Brianna Laugher wrote: > Hi, > > I just spent some time reading the dev docs so these comments are just > based on the docs and not actually using the new API. In general it > looks pretty sensible. > > - being able to have funcargs like funcargs directly is really nice, a > lot more obvious than calling request.getfuncargvalue('foo') > - why addfinalizer and not teardown? > -although I don't really know what cached_setup did, the trinity of > defining the scope, setup and teardown methods made sense to me. Now > the scope is in a decorator, the setup is implicitly the entire thing > that is happening and the teardown seems somewhat awkwardly tacked on. > - none of the "addfinalizer" examples take an argument, how would you > convert an old-style teardown method to that? e.g. we have a lot of > funcargs which do things like > return request.cached_setup(setup=setup, > teardown=lambda obj: obj.close(), > scope='function') > - Sometimes things are referred to as "funcargs", sometimes they are > referred to as "injected resources". Is there any difference here? The > funcarg is the actual function and the injected resource is the > instance in a specific test function? I suggest to use the term > "funcarg" as much as possible as it is specific and a necessary > concept for using pytest with any depth. > > Some of the following comments are fairly picky so feel free to ignore them. > > funcargs.txt > line 118 - I think in this first incarnation of the smtp funcarg > (factory? what to call it now?), it doesn't actually need to take a > testcontext, right? > > line 527 "Parametrizing test functions" - may be worth having a simple > example showing a combination using both (test data) parametrization > and funcarg parametrization, to emphasise how they are differently > useful. Using a database as an example of funcarg parametrization is > good, maybe better than values like 1/2. I feel like parametrizing > tests (test data) is probably the more common use case and it is a > little buried amongst the heavy duty parametrized funcargs. > > line 598 "Basic ``pytest_generate_tests`` example" - I think this is > not a very basic example! I think it is copied from parametrize.txt > page, where it might make more sense. Here is what I would consider a > basic example. > > # code > def isSquare(n): > n = n ** 0.5 > return int(n) == n > > # test file > def pytest_generate_tests(metafunc): > squares = [1, 4, 9, 16, 25, 36, 49] > for n in range(1, 50): > expected = n in squares > if metafunc.function.__name__ == 'test_isSquare': > metafunc.addcall(id=n, funcargs=dict(n=n, expected=expected)) > > > def test_isSquare(n, expected): > assert isSquare(n) == expected > > Well, this is so trivial you might bundle it all into a single test, > but it can be useful to have each case genuinely be a separate test. > You could also shoe-horn this data into a parametrize decorator I > suppose but it is nicer to have more space if your test data is more > complicated, to explain it. > > I am starting to have a feeling that the way my project has been using > generate_tests is not the way everyone else uses it. In our > conftest.py one of the enterprising developers on the project (who got > us all onto py.test initially) put this: > > def pytest_generate_tests(__multicall__, metafunc): > """Supports parametrised tests using generate_ fns. > Use multicall to call any other pytest_generate_tests hooks first. > If the test_ fn has a generate_ fn then call it with the metafunc > to let it parametrise the test. > """ > __multicall__.execute() > name = metafunc.function.__name__.replace('test_', 'generate_') > fn = getattr(metafunc.module, name, None) > if fn: > fn(metafunc) > > this means I can simplify my example above to: > > def generate_isSquare(metafunc): > squares = [1, 4, 9, 16, 25, 36, 49] > for n in range(1, 50): > expected = n in squares > metafunc.addcall(id=n, funcargs=dict(n=n, expected=expected)) > > maybe this is a useful pattern for others? or maybe we're doing it > wrong, I dunno :) > > in funcarg_compare.txt > line 53 - (in old-style) "4. there is no way how you can make use of > funcarg factories in xUnit setup methods." > > Is there an example of this? I would like to see it! This sounds like > a way for people with existing xUnit style test suites to more easily > convert to funcarg style. > > Maybe with all the parametrized test examples it is a good idea to > show the verbose output, just to be more explicit about what is being > generated. > > As an aside, there is a comment in example/markers.txt line 144 that > "You can use the ``-k`` command line option to only run tests with > names matching > the given argument". Whether it is an oversight in the functionality > or the docs, I have always found that -k also matches against markers > (py.test.mark decorators). (Currently using pytest 2.2.3) > > in setup.txt > line 65 'old = tmpdir.chdir()' > - it's not really relevant to the example but I don't know what this > does, and I couldn't find out, so it's just a bit distracting. > > thanks as always for providing this great library :) > > cheers > Brianna > > -- > They've just been waiting in a mountain for the right moment: > http://modernthings.org/ -- They've just been waiting in a mountain for the right moment: http://modernthings.org/ From issues-reply at bitbucket.org Mon Aug 20 16:25:26 2012 From: issues-reply at bitbucket.org (Ronny Pfannschmidt) Date: Mon, 20 Aug 2012 14:25:26 -0000 Subject: [py-dev] [hpk42/pytest] errors in factory setup call terminalwriter with bogous args (issue #179) Message-ID: <9c2e74f26693507a3826e3187c5cf958@bitbucket.org> --- you can reply above this line --- New issue 179: errors in factory setup call terminalwriter with bogous args https://bitbucket.org/hpk42/pytest/issue/179/errors-in-factory-setup-call Ronny Pfannschmidt: {{{ ============================= test session starts ============================== platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev10 plugins: xdist, codecheckers, couchdbkit, cache collecting ... collected 300 items tests/test_checkout.py EEEE tests/test_client.py EEEEE tests/test_metadata.py EEEEEE tests/test_util.py E tests/test_vc_script.py E tests/remote/test_remote.py EE tests/remote/test_remote_object.py EE tests/repository/test_commit_builder.py E tests/repository/test_creation.py EE tests/repository/test_find.py E tests/repository/test_history.py EEEEE tests/repository/test_history_diff.py E tests/repository/test_renames.py Ex tests/repository/test_repopath.py EE tests/repository/test_simple_renames.py x tests/workdir/test_common.py EEEEEEEEEEE tests/workdir/test_stated_path.py EEE tests/test_checkout.py EEEE tests/test_client.py EEEEE tests/test_metadata.py EEEEEE tests/test_util.py E tests/test_vc_script.py E tests/remote/test_remote.py EE tests/remote/test_remote_object.py EE tests/repository/test_commit_builder.py E tests/repository/test_creation.py EE tests/repository/test_find.py E tests/repository/test_history.py EEEEE tests/repository/test_history_diff.py E tests/repository/test_renames.py Ex tests/repository/test_repopath.py EE tests/repository/test_simple_renames.py x tests/workdir/test_common.py EEEEEEEEEEE tests/workdir/test_stated_path.py EEE tests/test_checkout.py EEEE tests/test_client.py EEEEE tests/test_metadata.py EEEEEE tests/test_util.py E tests/test_vc_script.py E tests/remote/test_remote.py EE tests/remote/test_remote_object.py EE tests/repository/test_commit_builder.py E tests/repository/test_creation.py EE tests/repository/test_find.py E tests/repository/test_history.py EEEEE tests/repository/test_history_diff.py E tests/repository/test_renames.py Ex tests/repository/test_repopath.py EE tests/repository/test_simple_renames.py x tests/workdir/test_common.py EEEEEEEEEEE tests/workdir/test_stated_path.py EEE tests/test_checkout.py EEEE tests/test_client.py EEEEE tests/test_metadata.py EEEEEE tests/test_util.py E tests/test_vc_script.py E tests/remote/test_remote.py EE tests/remote/test_remote_object.py EE tests/repository/test_commit_builder.py E tests/repository/test_creation.py EE tests/repository/test_find.py E tests/repository/test_history.py EEEEE tests/repository/test_history_diff.py E tests/repository/test_renames.py Ex tests/repository/test_repopath.py EE tests/repository/test_simple_renames.py x tests/workdir/test_common.py EEEEEEEEEEE tests/workdir/test_stated_path.py EEE tests/test_checkout.py EEEE tests/test_client.py EEEEE tests/test_metadata.py EEEEEE tests/test_util.py E tests/test_vc_script.py E tests/remote/test_remote.py EE tests/remote/test_remote_object.py EE tests/repository/test_commit_builder.py E tests/repository/test_creation.py EE tests/repository/test_find.py E tests/repository/test_history.py EEEEE tests/repository/test_history_diff.py E tests/repository/test_renames.py Ex tests/repository/test_repopath.py EE tests/repository/test_simple_renames.py x tests/workdir/test_common.py EEEEEEEEEEE tests/workdir/test_stated_path.py EEE tests/test_checkout.py EEEE tests/test_client.py EEEEE tests/test_metadata.py EEEEEE tests/test_util.py E tests/test_vc_script.py E tests/remote/test_remote.py EE tests/remote/test_remote_object.py EE tests/repository/test_commit_builder.py E tests/repository/test_creation.py EE tests/repository/test_find.py E tests/repository/test_history.py EEEEE tests/repository/test_history_diff.py E tests/repository/test_renames.py Ex tests/repository/test_repopath.py EE tests/repository/test_simple_renames.py x tests/workdir/test_common.py EEEEEEEEEEE tests/workdir/test_stated_path.py EEE ==================================== ERRORS ==================================== ___________ ERROR at setup of test_checkout_local[direct-subversion] ___________ def pytest_funcarg__repo(request, mgr): LookupError: no factory found for argument 'mgr' available funcargs: pytestconfig, couchdb, couchdb_server, recwarn, spec, repo, wd, monkeypatch, vcs, capfd, capsys, tmpdir, backend use 'py.test --funcargs [testpath]' for help on them. /home/ronny/Projects/pissoff/anyvc/tests/conftest.py:105 ____________ ERROR at setup of test_clone_local[direct-subversion] _____________ def pytest_funcarg__wd(request, mgr): LookupError: no factory found for argument 'mgr' available funcargs: pytestconfig, couchdb, couchdb_server, recwarn, spec, repo, wd, monkeypatch, vcs, capfd, capsys, tmpdir, backend use 'py.test --funcargs [testpath]' for help on them. /home/ronny/Projects/pissoff/anyvc/tests/conftest.py:111 ____________ ERROR at setup of test_clone_remote[direct-subversion] ____________ @py.test.mark.feature('wd:heavy') def test_clone_remote(mgr): LookupError: no factory found for argument 'mgr' available funcargs: pytestconfig, couchdb, couchdb_server, recwarn, spec, repo, wd, monkeypatch, vcs, capfd, capsys, tmpdir, backend use 'py.test --funcargs [testpath]' for help on them. /home/ronny/Projects/pissoff/anyvc/tests/test_checkout.py:37 __________ ERROR at setup of test_checkout_remote[direct-subversion] ___________ @pytest.factory(scope='session') def backend(request, vcs, spec): LookupError: no factory found for argument 'request' available funcargs: pytestconfig, couchdb, couchdb_server, recwarn, spec, repo, wd, monkeypatch, vcs, capfd, capsys, tmpdir, backend use 'py.test --funcargs [testpath]' for help on them. /home/ronny/Projects/pissoff/anyvc/tests/conftest.py:72 ________ ERROR at setup of test_output_state[direct-subversion-deleted] ________ <_pytest.python.SetupCall instance at 0x14253f8>Traceback (most recent call last): File "/home/ronny/.local/virtualenvs/tools/bin/py.test", line 9, in load_entry_point('pytest==2.3.0.dev10', 'console_scripts', 'py.test')() File "/home/ronny/Projects/codespeak/pytest/_pytest/core.py", line 470, in main exitstatus = config.hook.pytest_cmdline_main(config=config) File "/home/ronny/Projects/codespeak/pytest/_pytest/core.py", line 422, in __call__ return self._docall(methods, kwargs) File "/home/ronny/Projects/codespeak/pytest/_pytest/core.py", line 433, in _docall res = mc.execute() File "/home/ronny/Projects/codespeak/pytest/_pytest/core.py", line 351, in execute res = method(**kwargs) File "/home/ronny/Projects/codespeak/pytest/_pytest/main.py", line 103, in pytest_cmdline_main return wrap_session(config, _main) File "/home/ronny/Projects/codespeak/pytest/_pytest/main.py", line 95, in wrap_session exitstatus=session.exitstatus or (session._testsfailed and 1)) File "/home/ronny/Projects/codespeak/pytest/_pytest/core.py", line 422, in __call__ return self._docall(methods, kwargs) File "/home/ronny/Projects/codespeak/pytest/_pytest/core.py", line 433, in _docall res = mc.execute() File "/home/ronny/Projects/codespeak/pytest/_pytest/core.py", line 351, in execute res = method(**kwargs) File "/home/ronny/Projects/codespeak/pytest/_pytest/terminal.py", line 332, in pytest_sessionfinish self.summary_errors() File "/home/ronny/Projects/codespeak/pytest/_pytest/terminal.py", line 430, in summary_errors self._outrep_summary(rep) File "/home/ronny/Projects/codespeak/pytest/_pytest/terminal.py", line 433, in _outrep_summary rep.toterminal(self._tw) File "/home/ronny/Projects/codespeak/pytest/_pytest/runner.py", line 155, in toterminal longrepr.toterminal(out) File "/home/ronny/Projects/codespeak/pytest/_pytest/python.py", line 1215, in toterminal tw.line(line) File "/home/ronny/Projects/codespeak/py/py/_io/terminalwriter.py", line 185, in line self._checkfill(s) File "/home/ronny/Projects/codespeak/py/py/_io/terminalwriter.py", line 197, in _checkfill diff2last = self._lastlen - len(line) AttributeError: SetupCall instance has no attribute '__len__' }}} -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-reply at bitbucket.org Wed Aug 22 14:35:59 2012 From: issues-reply at bitbucket.org (Ralf Schmitt) Date: Wed, 22 Aug 2012 12:35:59 -0000 Subject: [py-dev] [hpk42/pytest] xdist hides errors (issue #180) Message-ID: <82db8da2e944026ba3aa9b00e99a9d58@bitbucket.org> --- you can reply above this line --- New issue 180: xdist hides errors https://bitbucket.org/hpk42/pytest/issue/180/xdist-hides-errors Ralf Schmitt: when running without xdist I get the following error: $ py.test ======================================= test session starts ======================================= platform linux2 -- Python 2.7.3 -- pytest-2.2.4 collected 0 items / 1 errors ============================================= ERRORS ============================================== _______________________________________ ERROR collecting . ________________________________________ ../ve/local/lib/python2.7/site-packages/py/_path/common.py:315: in visit > for x in Visitor(fil, rec, ignore, bf, sort).gen(self): ../ve/local/lib/python2.7/site-packages/py/_path/common.py:351: in gen > if p.check(dir=1) and (rec is None or rec(p))]) ../ve/local/lib/python2.7/site-packages/_pytest/main.py:484: in _recurse > ihook.pytest_collect_directory(path=path, parent=self) ../ve/local/lib/python2.7/site-packages/_pytest/main.py:140: in call_matching_hooks > plugins = self.config._getmatchingplugins(self.fspath) ../ve/local/lib/python2.7/site-packages/_pytest/config.py:289: in _getmatchingplugins > plugins += self._conftest.getconftestmodules(fspath) ../ve/local/lib/python2.7/site-packages/_pytest/config.py:188: in getconftestmodules > clist.append(self.importconftest(conftestpath)) ../ve/local/lib/python2.7/site-packages/_pytest/config.py:224: in importconftest > self._postimport(mod) ../ve/local/lib/python2.7/site-packages/_pytest/config.py:229: in _postimport > self._onimport(mod) ../ve/local/lib/python2.7/site-packages/_pytest/config.py:278: in _onimportconftest > self.pluginmanager.consider_conftest(conftestmodule) ../ve/local/lib/python2.7/site-packages/_pytest/core.py:179: in consider_conftest > if self.register(conftestmodule, name=conftestmodule.__file__): ../ve/local/lib/python2.7/site-packages/_pytest/core.py:89: in register > self.hook.pytest_plugin_registered(manager=self, plugin=plugin) ../ve/local/lib/python2.7/site-packages/_pytest/core.py:421: in __call__ > return self._docall(methods, kwargs) ../ve/local/lib/python2.7/site-packages/_pytest/core.py:432: in _docall > res = mc.execute() ../ve/local/lib/python2.7/site-packages/_pytest/core.py:350: in execute > res = method(**kwargs) ../ve/local/lib/python2.7/site-packages/_pytest/core.py:231: in pytest_plugin_registered > {'config': self._config}) ../ve/local/lib/python2.7/site-packages/_pytest/core.py:317: in call_plugin > kwargs=kwargs, firstresult=True).execute() ../ve/local/lib/python2.7/site-packages/_pytest/core.py:350: in execute > res = method(**kwargs) easyindex/conftest.py:75: in pytest_configure > build_index() easyindex/conftest.py:55: in build_index > index = EasyIndex.Index(indexname + ".tmp") easyindex/EasyIndex/__init__.py:45: in Index > from EasyIndex.core import Index easyindex/EasyIndex/core.py:10: in > import _pyfilter E ImportError: No module named _ezindex ----------------------------------------- Captured stdout ----------------------------------------- [building initial index in /home/ralf/bbot/easyindex/tmp_dummy_test_index... ===================================== 1 error in 0.08 seconds ===================================== Running with py.test -n6 gives: $ py.test -n6 ======================================= test session starts ======================================= platform linux2 -- Python 2.7.3 -- pytest-2.2.4 gw0 [0] / gw1 [0] / gw2 [0] / gw3 [0] / gw4 [0] / gw5 [0] scheduling tests via LoadScheduling ======================================== in 0.54 seconds ========================================= xdist hides the error in that case! -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From brianna.laugher at gmail.com Thu Aug 23 05:39:12 2012 From: brianna.laugher at gmail.com (Brianna Laugher) Date: Thu, 23 Aug 2012 13:39:12 +1000 Subject: [py-dev] pytest talk Message-ID: Hi, So I gave my talk on pytest at PyCon AU on the weekend :) Thought it might be of interest to folks here. info: http://2012.pycon-au.org/schedule/52/view_talk?day=sunday slides: http://www.slideshare.net/pfctdayelise/funcargs-other-fun-with-pytest code: https://gist.github.com/3386951 video: http://www.youtube.com/watch?v=DTNejE9EraI cheers Brianna -- They've just been waiting in a mountain for the right moment: http://modernthings.org/ From holger at merlinux.eu Sun Aug 26 16:30:09 2012 From: holger at merlinux.eu (holger krekel) Date: Sun, 26 Aug 2012 14:30:09 +0000 Subject: [py-dev] pytest talk In-Reply-To: References: Message-ID: <20120826143009.GU26019@merlinux.eu> Hi Brianna, On Thu, Aug 23, 2012 at 13:39 +1000, Brianna Laugher wrote: > Hi, > > So I gave my talk on pytest at PyCon AU on the weekend :) Thought it > might be of interest to folks here. > > info: http://2012.pycon-au.org/schedule/52/view_talk?day=sunday > slides: http://www.slideshare.net/pfctdayelise/funcargs-other-fun-with-pytest > code: https://gist.github.com/3386951 > video: http://www.youtube.com/watch?v=DTNejE9EraI thanks for the links and the talk, just watched it and like it! I took the freedom to mention it in the "talks" page to be online with the new release: http://pytest.org/dev/talks.html best, holger > cheers > Brianna > > -- > They've just been waiting in a mountain for the right moment: > http://modernthings.org/ > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From holger at merlinux.eu Sun Aug 26 16:54:35 2012 From: holger at merlinux.eu (holger krekel) Date: Sun, 26 Aug 2012 14:54:35 +0000 Subject: [py-dev] new resource API documentation comments In-Reply-To: References: Message-ID: <20120826145435.GV26019@merlinux.eu> Hi Brianna, On Thu, Aug 16, 2012 at 18:58 +1000, Brianna Laugher wrote: > Hi, > > I just spent some time reading the dev docs so these comments are just > based on the docs and not actually using the new API. In general it > looks pretty sensible. thanks for your time and your feedback, I appreciate it! > - being able to have funcargs like funcargs directly is really nice, a > lot more obvious than calling request.getfuncargvalue('foo') agreed :) > - why addfinalizer and not teardown? The old-style pytest_funcarg__NAME(request) api offered a request.addfinalizer() function already so i wanted to carry on with this name. I also considered addcleanup() similar to the python unittest package and am open to other naming suggestions before the september release. > -although I don't really know what cached_setup did, the trinity of > defining the scope, setup and teardown methods made sense to me. Now > the scope is in a decorator, the setup is implicitly the entire thing > that is happening and the teardown seems somewhat awkwardly tacked on. I see that point. In principle we could have the factory decorator take a "teardown" parameter and it would receive the factory-created value. I'd find it a bit awkward to advertise first naming a teardown before even stating the setup code, though. And it wouldn't help the asymetry you are observing. > - none of the "addfinalizer" examples take an argument, how would you > convert an old-style teardown method to that? e.g. we have a lot of > funcargs which do things like > return request.cached_setup(setup=setup, > teardown=lambda obj: obj.close(), > scope='function') In a factory-function you will create the value to return at some point and then you can use it from the finalizer, like so: ... val = createval() def fin(): uncreate(val) testcontext.addfinalizer(fin) > - Sometimes things are referred to as "funcargs", sometimes they are > referred to as "injected resources". Is there any difference here? The > funcarg is the actual function and the injected resource is the > instance in a specific test function? I suggest to use the term > "funcarg" as much as possible as it is specific and a necessary > concept for using pytest with any depth. I agree. In an early draft a radically different approach which used the term "resources" was considered. Due to feedback similar to yours I eventually went with using "funcargs" again. A funcarg is what appears in a test or setup function as an argument, it also _is_ a resource. I am open to reduce usages of "resource" if that is confusing. > Some of the following comments are fairly picky so feel free to ignore them. > > funcargs.txt > line 118 - I think in this first incarnation of the smtp funcarg > (factory? what to call it now?), it doesn't actually need to take a > testcontext, right? fixed. > line 527 "Parametrizing test functions" - may be worth having a simple > example showing a combination using both (test data) parametrization > and funcarg parametrization, to emphasise how they are differently > useful. Using a database as an example of funcarg parametrization is > good, maybe better than values like 1/2. I feel like parametrizing > tests (test data) is probably the more common use case and it is a > little buried amongst the heavy duty parametrized funcargs. I think with test function parametrization the parametrized funcargs usually relate to the particular test and the parameters are simple objects. Objects such as Databases are more complex resources and parametrization is then best defined at factory level. Adding a "mixed" example makes sense. > line 598 "Basic ``pytest_generate_tests`` example" - I think this is > not a very basic example! I think it is copied from parametrize.txt > page, where it might make more sense. Here is what I would consider a > basic example. > > # code > def isSquare(n): > n = n ** 0.5 > return int(n) == n > > # test file > def pytest_generate_tests(metafunc): > squares = [1, 4, 9, 16, 25, 36, 49] > for n in range(1, 50): > expected = n in squares > if metafunc.function.__name__ == 'test_isSquare': > metafunc.addcall(id=n, funcargs=dict(n=n, expected=expected)) > > > def test_isSquare(n, expected): > assert isSquare(n) == expected > > Well, this is so trivial you might bundle it all into a single test, > but it can be useful to have each case genuinely be a separate test. > You could also shoe-horn this data into a parametrize decorator I > suppose but it is nicer to have more space if your test data is more > complicated, to explain it. OK, I consider adding this when i tackle another doc refactoring. thanks for providing it. > I am starting to have a feeling that the way my project has been using > generate_tests is not the way everyone else uses it. In our > conftest.py one of the enterprising developers on the project (who got > us all onto py.test initially) put this: > > def pytest_generate_tests(__multicall__, metafunc): > """Supports parametrised tests using generate_ fns. > Use multicall to call any other pytest_generate_tests hooks first. > If the test_ fn has a generate_ fn then call it with the metafunc > to let it parametrise the test. > """ > __multicall__.execute() > name = metafunc.function.__name__.replace('test_', 'generate_') > fn = getattr(metafunc.module, name, None) > if fn: > fn(metafunc) > > this means I can simplify my example above to: > > def generate_isSquare(metafunc): > squares = [1, 4, 9, 16, 25, 36, 49] > for n in range(1, 50): > expected = n in squares > metafunc.addcall(id=n, funcargs=dict(n=n, expected=expected)) > > maybe this is a useful pattern for others? or maybe we're doing it > wrong, I dunno :) It is a good example for implementing a custom scheme and making the life of test writers (who do not need to care about the helper in conftest.py) easier. Personally, i'd prefer the decorator approach but it all depends a bit on the exact use case. > in funcarg_compare.txt > line 53 - (in old-style) "4. there is no way how you can make use of > funcarg factories in xUnit setup methods." > > Is there an example of this? I would like to see it! This sounds like > a way for people with existing xUnit style test suites to more easily > convert to funcarg style. There are some preliminary docs about it: http://pytest.org/dev/setup.html The examples there use funcargs in setup functions just as you use them in test function. Or am i misunderstanding your question? (you seem to know about the setup docs already, judging from below). > Maybe with all the parametrized test examples it is a good idea to > show the verbose output, just to be more explicit about what is being > generated. > > As an aside, there is a comment in example/markers.txt line 144 that > "You can use the ``-k`` command line option to only run tests with > names matching > the given argument". Whether it is an oversight in the functionality > or the docs, I have always found that -k also matches against markers > (py.test.mark decorators). (Currently using pytest 2.2.3) It was intended so the docs are imprecise there. I take it you also know about "-m", btw. > in setup.txt > line 65 'old = tmpdir.chdir()' > - it's not really relevant to the example but I don't know what this > does, and I couldn't find out, so it's just a bit distracting. > > thanks as always for providing this great library :) you are most welcome and thanks for your feedback! best, holger > cheers > Brianna > > -- > They've just been waiting in a mountain for the right moment: > http://modernthings.org/ > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From holger at merlinux.eu Sun Aug 26 16:57:00 2012 From: holger at merlinux.eu (holger krekel) Date: Sun, 26 Aug 2012 14:57:00 +0000 Subject: [py-dev] new resource API documentation comments In-Reply-To: References: Message-ID: <20120826145700.GW26019@merlinux.eu> On Fri, Aug 17, 2012 at 17:32 +1000, Brianna Laugher wrote: > Also, > > Would this be a roughly equivalent old-style to the smtp examples in > http://pytest.org/dev/funcargs.html ? > > def pytest_funcarg__smtpMerLinux(request): > smtp = smtplib.SMTP("merlinux.eu") > def teardown(smtp): > print ("finalizing %s" % smtp) > smtp.close() > return request.cached_setup(setup=lambda: smtp, teardown=teardown, > scope='session') > > > def pytest_funcarg__smtpMailPython(request): > smtp = smtplib.SMTP("mail.python.org") > def teardown(smtp): > print ("finalizing %s" % smtp) > smtp.close() > return request.cached_setup(setup=lambda: smtp, teardown=teardown, > scope='session') > > # test file > def pytest_generate_tests(metafunc): > merlinux = request.getfuncargvalue('smtpMerLinux') > mailpython = request.getfuncargvalue('smtpMailPython') > > if'smtp' in metafunc.funcargnames: > metafunc.addcall(id='merlinux.eu', param=merlinux) # ? would > this work? seems magic > metafunc.addcall(id='mail.python.org', param=mailpython) > > I feel like this makes it clear how much more powerful parametrizing > funcargs themselves is. The old-style code would even be a bit more involved. I guess it could be added to http://pytest.org/dev/funcarg_compare.html I wouldn't put it to the entry-level main docs because for newcomers it's not immediately neccessary to know the old ways :) best, holger > cheers > Brianna > > > > > On 16 August 2012 18:58, Brianna Laugher wrote: > > Hi, > > > > I just spent some time reading the dev docs so these comments are just > > based on the docs and not actually using the new API. In general it > > looks pretty sensible. > > > > - being able to have funcargs like funcargs directly is really nice, a > > lot more obvious than calling request.getfuncargvalue('foo') > > - why addfinalizer and not teardown? > > -although I don't really know what cached_setup did, the trinity of > > defining the scope, setup and teardown methods made sense to me. Now > > the scope is in a decorator, the setup is implicitly the entire thing > > that is happening and the teardown seems somewhat awkwardly tacked on. > > - none of the "addfinalizer" examples take an argument, how would you > > convert an old-style teardown method to that? e.g. we have a lot of > > funcargs which do things like > > return request.cached_setup(setup=setup, > > teardown=lambda obj: obj.close(), > > scope='function') > > - Sometimes things are referred to as "funcargs", sometimes they are > > referred to as "injected resources". Is there any difference here? The > > funcarg is the actual function and the injected resource is the > > instance in a specific test function? I suggest to use the term > > "funcarg" as much as possible as it is specific and a necessary > > concept for using pytest with any depth. > > > > Some of the following comments are fairly picky so feel free to ignore them. > > > > funcargs.txt > > line 118 - I think in this first incarnation of the smtp funcarg > > (factory? what to call it now?), it doesn't actually need to take a > > testcontext, right? > > > > line 527 "Parametrizing test functions" - may be worth having a simple > > example showing a combination using both (test data) parametrization > > and funcarg parametrization, to emphasise how they are differently > > useful. Using a database as an example of funcarg parametrization is > > good, maybe better than values like 1/2. I feel like parametrizing > > tests (test data) is probably the more common use case and it is a > > little buried amongst the heavy duty parametrized funcargs. > > > > line 598 "Basic ``pytest_generate_tests`` example" - I think this is > > not a very basic example! I think it is copied from parametrize.txt > > page, where it might make more sense. Here is what I would consider a > > basic example. > > > > # code > > def isSquare(n): > > n = n ** 0.5 > > return int(n) == n > > > > # test file > > def pytest_generate_tests(metafunc): > > squares = [1, 4, 9, 16, 25, 36, 49] > > for n in range(1, 50): > > expected = n in squares > > if metafunc.function.__name__ == 'test_isSquare': > > metafunc.addcall(id=n, funcargs=dict(n=n, expected=expected)) > > > > > > def test_isSquare(n, expected): > > assert isSquare(n) == expected > > > > Well, this is so trivial you might bundle it all into a single test, > > but it can be useful to have each case genuinely be a separate test. > > You could also shoe-horn this data into a parametrize decorator I > > suppose but it is nicer to have more space if your test data is more > > complicated, to explain it. > > > > I am starting to have a feeling that the way my project has been using > > generate_tests is not the way everyone else uses it. In our > > conftest.py one of the enterprising developers on the project (who got > > us all onto py.test initially) put this: > > > > def pytest_generate_tests(__multicall__, metafunc): > > """Supports parametrised tests using generate_ fns. > > Use multicall to call any other pytest_generate_tests hooks first. > > If the test_ fn has a generate_ fn then call it with the metafunc > > to let it parametrise the test. > > """ > > __multicall__.execute() > > name = metafunc.function.__name__.replace('test_', 'generate_') > > fn = getattr(metafunc.module, name, None) > > if fn: > > fn(metafunc) > > > > this means I can simplify my example above to: > > > > def generate_isSquare(metafunc): > > squares = [1, 4, 9, 16, 25, 36, 49] > > for n in range(1, 50): > > expected = n in squares > > metafunc.addcall(id=n, funcargs=dict(n=n, expected=expected)) > > > > maybe this is a useful pattern for others? or maybe we're doing it > > wrong, I dunno :) > > > > in funcarg_compare.txt > > line 53 - (in old-style) "4. there is no way how you can make use of > > funcarg factories in xUnit setup methods." > > > > Is there an example of this? I would like to see it! This sounds like > > a way for people with existing xUnit style test suites to more easily > > convert to funcarg style. > > > > Maybe with all the parametrized test examples it is a good idea to > > show the verbose output, just to be more explicit about what is being > > generated. > > > > As an aside, there is a comment in example/markers.txt line 144 that > > "You can use the ``-k`` command line option to only run tests with > > names matching > > the given argument". Whether it is an oversight in the functionality > > or the docs, I have always found that -k also matches against markers > > (py.test.mark decorators). (Currently using pytest 2.2.3) > > > > in setup.txt > > line 65 'old = tmpdir.chdir()' > > - it's not really relevant to the example but I don't know what this > > does, and I couldn't find out, so it's just a bit distracting. > > > > thanks as always for providing this great library :) > > > > cheers > > Brianna > > > > -- > > They've just been waiting in a mountain for the right moment: > > http://modernthings.org/ > > > > -- > They've just been waiting in a mountain for the right moment: > http://modernthings.org/ > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From issues-reply at bitbucket.org Tue Aug 28 12:26:33 2012 From: issues-reply at bitbucket.org (arigo) Date: Tue, 28 Aug 2012 10:26:33 -0000 Subject: [py-dev] [hpk42/pytest] --pdb does not work when collecting (issue #181) Message-ID: <7414f176a31a5c6774e5230dbd7ad84a@bitbucket.org> --- you can reply above this line --- New issue 181: --pdb does not work when collecting https://bitbucket.org/hpk42/pytest/issue/181/pdb-does-not-work-when-collecting arigo: "py.test --pdb" ignores the --pdb option for crashes that occur during collect. Try to run "py.test --pdb test_foo.py" with only the line "crash_here" in test_foo.py. -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-reply at bitbucket.org Tue Aug 28 12:41:34 2012 From: issues-reply at bitbucket.org (Ronny Pfannschmidt) Date: Tue, 28 Aug 2012 10:41:34 -0000 Subject: [py-dev] [hpk42/pytest] pytester.inprocess_run ignores passed plugins (issue #182) Message-ID: --- you can reply above this line --- New issue 182: pytester.inprocess_run ignores passed plugins https://bitbucket.org/hpk42/pytest/issue/182/pytesterinprocess_run-ignores-passed Ronny Pfannschmidt: -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-reply at bitbucket.org Tue Aug 28 13:15:41 2012 From: issues-reply at bitbucket.org (Ronny Pfannschmidt) Date: Tue, 28 Aug 2012 11:15:41 -0000 Subject: [py-dev] [hpk42/pytest] investigate armin rigos conftest (issue #183) Message-ID: <5a8090b4a6a53e017cc562b009e55ae7@bitbucket.org> --- you can reply above this line --- New issue 183: investigate armin rigos conftest https://bitbucket.org/hpk42/pytest/issue/183/investigate-armin-rigos-conftest Ronny Pfannschmidt: armin grew his conftest quite some amounts to keep output semilar to what it was in 2007 (pre 2.x) so we should investigate what was better back then wrt output and bring those things back as builtins or plugins -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-reply at bitbucket.org Tue Aug 28 17:24:16 2012 From: issues-reply at bitbucket.org (Ronny Pfannschmidt) Date: Tue, 28 Aug 2012 15:24:16 -0000 Subject: [py-dev] [hpk42/pytest] usage of py.code.Source.deindent sometimes strangely hangs (issue #184) Message-ID: <9edd6d503974332249d6942742feeaae@bitbucket.org> --- you can reply above this line --- New issue 184: usage of py.code.Source.deindent sometimes strangely hangs https://bitbucket.org/hpk42/pytest/issue/184/usage-of-pycodesourcedeindent-sometimes Ronny Pfannschmidt: the effect is rather strange, the orginal deindent hangs, same goes for one replaced with textwrap.dedent however if one interupts and jumps into pdb, then runs the same dedent again, its suddenly fast the effect can be seen on pypy commit 09c187ceba4a by testing ` jit/backend/llgraph/test -k test_jump --fulltrace` -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.