From holger at merlinux.eu Wed Jun 1 09:49:13 2011 From: holger at merlinux.eu (holger krekel) Date: Wed, 1 Jun 2011 07:49:13 +0000 Subject: [py-dev] Decorators and funcargs in py.test In-Reply-To: <1306787910.3360.15.camel@Klappe2> References: <1306785315.3360.1.camel@Klappe2> <1306787910.3360.15.camel@Klappe2> Message-ID: <20110601074913.GE20287@merlinux.eu> Hi Vyacheslav, hi Ronny, On Mon, May 30, 2011 at 22:38 +0200, Ronny Pfannschmidt wrote: > On Mon, 2011-05-30 at 16:23 -0400, Vyacheslav Rafalskiy wrote: > > No problem. Here is my (real life) example. > > > > My functional test functions may or may not return for different > > reasons (like a faulty web application or middleware). I want to > > declare a fail if it takes more than so many seconds to complete. So I > > write a decorator > > run_with_timeout(), which will start the function in a new thread and > > abandon it after timeout. > > This can easily be solved by combining something like > pytest.mark('timeout') and a override to pytest_pyfunc_call using it I agree. Vyacheslav, if you can't make sense of the "use pytest.mark()" and provide-a-hook suggestion, one of us will be certainly be happy to provide a more complete example and add it to the docs. > i suppose this could also take a look at extending the "--boxed" mode > (which forks for each test and uses subprocesses but doesn?t handle > timeouts atm. this is part of the pytest-xdist plugin, however. Question is, if we shouldn't eventually grow timeout-support in core pytest through "alarm" or so. best, holger > > > > A I stated in OP, it is not that I cannot do it. I can and I do. My > > point is that it makes sense to allow decorators and should not be > > very difficult (see the example in OP). > > personally i am opposed to creating new data conventions for problems > that can be solved with plain marks + a hook > > usually there are 2 reasons people use to decorate tests > > my opinions for implementing those are > a) add arguments + their cleanups -> funcargs please, they are made for > that > b) use more sophisticated call's -> hooks please, maybe add a bug ticket > for empowering one to return a exception-info, so stuff like > thread-wrappers can pass that more nicely for failures > > *i* really think it is wrong to decorate test functions that way and > expect stuff to work > > there are already plenty of mechanisms to change the behavior of pytest > test function calling in the desired way, none of those require hacks to > pass around argspecs > > -- Ronny > > > > Thanks, > > Vyacheslav > > > > On Mon, May 30, 2011 at 3:55 PM, Ronny Pfannschmidt > > wrote: > > > Hi, > > > > > > can you try to explain the usecase those decorators are fulfilling, > > > > > > there may be a better integrated way using pytest.mark + setup/teardown > > > hooks > > > > > > -- Ronny > > > > > > On Mon, 2011-05-30 at 15:17 -0400, Vyacheslav Rafalskiy wrote: > > >> Hi Holger, > > >> > > >> I am trying to make decorators work with test functions, which depend > > >> on funcargs. As it stands, they don't. > > >> Decorated functions lose funcargs. A workaround would be to decorate > > >> an internal function like this: > > >> > > >> def test_it(funcarg_it): > > >> @decorate_it > > >> def _test_it(): > > >> # test it > > >> > > >> _test_it() > > >> > > >> This works, but it is not nice. I'd rather wrote a decorator like > > >> > > >> def decorate_it(f): > > >> def _wrap_it(*args, **kwargs): > > >> # wrap f() here > > >> > > >> _wrap_it._varnames = _pytest.core.varnames(f) > > >> return _wrap_it > > >> > > >> and apply it straight to the test function. > > >> > > >> After examining the source code I even expected it to just work > > >> (magically of course) but it didn't. > > >> Do you think it is worthwhile? If so I can enter a feature request. > > >> > > >> Thanks, > > >> Vyacheslav > > >> _______________________________________________ > > >> py-dev mailing list > > >> py-dev at codespeak.net > > >> http://codespeak.net/mailman/listinfo/py-dev > > > > > > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev From holger at merlinux.eu Wed Jun 1 09:57:43 2011 From: holger at merlinux.eu (holger krekel) Date: Wed, 1 Jun 2011 07:57:43 +0000 Subject: [py-dev] setup_directory / xUnit extension? Re: migration from 1.3 to 2.0 In-Reply-To: References: <20110407061715.GE16231@merlinux.eu> <20110430142225.GY4071@merlinux.eu> <20110503094814.GU4071@merlinux.eu> <20110529074327.GX20287@merlinux.eu> <20110529075938.GY20287@merlinux.eu> Message-ID: <20110601075743.GF20287@merlinux.eu> On Mon, May 30, 2011 at 14:24 -0400, Vyacheslav Rafalskiy wrote: > On Sun, May 29, 2011 at 3:59 AM, holger krekel wrote: > > On Sun, May 29, 2011 at 07:43 +0000, holger krekel wrote: > >> > > good line of reasoning. ?It's mostly my intuition making me hesitant > >> > > to add setup_package like you suggest. ?And i wonder what it is about :) > >> > > Maybe it's that the root namespace of a test directory is often called > >> > > "testing" or "tests" ?(the "test" one is taken by Python stdlib already). > >> > > And therefore you would end up having to do "import testing" and > >> > > then use global state with something like "testing.STATE". > >> > > But i guess this doesn't look so bad to you, does it? :) > >> > > (The plugin/conftest system is designed such that it hardly > >> > > requires any imports to manage test state.) > >> > > > >> > > Any more opinions on setup_package(package)? If others find it useful > >> > > as well, i will consider introducing it with pytest-2.1. > >> > > >> > I guess I will have to withdraw the idea. Having to explicitly import > >> > the test package does not look nice at all. > >> > conftest.py rules! > >> > > >> > As to the two alternatives above I'd rather use > >> > pytest_setup_testloop(config) with direct access to config. > >> > >> I am now pondering following your original intention and introduce a > >> "setup_directory" to be put into conftest.py files. ?You could then > >> access the config object via pytest.config. Would that make sense > >> to you as well? > > > > To elaborate a wee bit: > > > > * setup_directory would be guranteed to be called for any test > > ?(both python, doctest or other test) within the directory hierarchy > > ?of the conftest.py dir and before any setup_module/class etc. is called. > > > > * teardown_directory would be guranteed to be called when a test > > ?is run that is not in the directory hierarchy. > > > > * if neccessary one can have setup_directory push test related > > ?state to some global module (from which tests could import for example) > > > > i am not yet sure about the idea but i guess it would be somewhat natural > > and complete the setup_*/teardown_* xUnit style fixture methods. > > > > holger > > Sounds great to me. Not much to add. Perhaps a parameter, say > directory_config, which > can then be imported by a test module like for example > > from pytest import directory_config > > This object could have different meaning depending on the directory > where the test module resides. This suggestion would replicate funcarg-functionality which allows to provide different/adapted resources to test modules from conftest.py files. I'd rather like to think about generally giving access to funcarg resources from xUnit setup functions which would enlarge what you can do from xUnit fixtures. Meanwhile, we could maybe introduce a global "pytest.appdata" object which can be used for passing state between various places in test like setup_directory or hooks and test methods or other setup methods. Several python packages (particularly web stuff) have global data anyway and maybe it's best to provide one managed storage for it. best, holger > > Vyacheslav > From rafalskiy at gmail.com Wed Jun 1 18:31:06 2011 From: rafalskiy at gmail.com (Vyacheslav Rafalskiy) Date: Wed, 1 Jun 2011 12:31:06 -0400 Subject: [py-dev] Decorators and funcargs in py.test In-Reply-To: <20110601074913.GE20287@merlinux.eu> References: <1306785315.3360.1.camel@Klappe2> <1306787910.3360.15.camel@Klappe2> <20110601074913.GE20287@merlinux.eu> Message-ID: Gentlemen! Thanks for your input. I really appreciate it. @Ronny 1. I do agree that creating a new data convention is a big deal. What I tried to suggest is using something that already exists in _pytest.core.varnames(). A comment in _pytest.python.getfuncargnames() indicates that the two functions may merge. If they do, and towards the former, my original suggestion will *just work*. 2. Using decorators is a great deal for me. They are powerful and expressive. I understand however the reluctance to open a can of worms here. The replacement using pytest.mark below seems acceptable to me. It still uses the same decorator function, just at a different place. @Holger, Ronny Here is what I came up with: #---------->> in test_1.py @pytest.mark.timeout(10) def test_f1(): # test here #---------->> in conftest.py def pytest_runtest_call(item): if hasattr(item.obj, 'timeout'): timeout = item.obj.timeout.args[0] item.obj = run_with_timeout(timeout)(item.obj) Your comments are welcome. Regards, Vyacheslav On Wed, Jun 1, 2011 at 3:49 AM, holger krekel wrote: > Hi Vyacheslav, hi Ronny, > > On Mon, May 30, 2011 at 22:38 +0200, Ronny Pfannschmidt wrote: >> On Mon, 2011-05-30 at 16:23 -0400, Vyacheslav Rafalskiy wrote: >> > No problem. Here is my (real life) example. >> > >> > My functional test functions may or may not return for different >> > reasons (like a faulty web application or middleware). I want to >> > declare a fail if it takes more than so many seconds to complete. So I >> > write a decorator >> > run_with_timeout(), which will start the function in a new thread and >> > abandon it after timeout. >> >> This can easily be solved by combining something like >> pytest.mark('timeout') and a override to pytest_pyfunc_call using it > > I agree. > > Vyacheslav, if you can't make sense of the "use pytest.mark()" and > provide-a-hook suggestion, one of us will be certainly be happy to > provide a more complete example and add it to the docs. > >> i suppose this could also take a look at extending the "--boxed" mode >> (which forks for each test and uses subprocesses but doesn?t handle >> timeouts atm. > > this is part of the pytest-xdist plugin, however. ?Question is, > if we shouldn't eventually grow timeout-support in core pytest > through "alarm" or so. > > best, > holger > >> > >> > A I stated in OP, it is not that I cannot do it. I can and I do. My >> > point is that it makes sense to allow decorators and should not be >> > very difficult (see the example in OP). >> >> personally i am opposed to creating new data conventions for problems >> that can be solved with plain marks + a hook >> >> usually there are 2 reasons people use to decorate tests >> >> my opinions for implementing those are >> a) add arguments + their cleanups -> funcargs please, they are made for >> that >> b) use more sophisticated call's -> hooks please, maybe add a bug ticket >> for empowering one to return a exception-info, so stuff like >> thread-wrappers can pass that more nicely for failures >> >> *i* really think it is wrong to decorate test functions that way and >> expect stuff to work >> >> there are already plenty of mechanisms to change the behavior of pytest >> test function calling in the desired way, none of those require hacks to >> pass around argspecs >> >> -- Ronny >> > >> > Thanks, >> > Vyacheslav >> > >> > On Mon, May 30, 2011 at 3:55 PM, Ronny Pfannschmidt >> > wrote: >> > > Hi, >> > > >> > > can you try to explain the usecase those decorators are fulfilling, >> > > >> > > there may be a better integrated way using pytest.mark + setup/teardown >> > > hooks >> > > >> > > -- Ronny >> > > >> > > On Mon, 2011-05-30 at 15:17 -0400, Vyacheslav Rafalskiy wrote: >> > >> Hi Holger, >> > >> >> > >> I am trying to make decorators work with test functions, which depend >> > >> on funcargs. As it stands, they don't. >> > >> Decorated functions lose funcargs. A workaround would be to decorate >> > >> an internal function like this: >> > >> >> > >> def test_it(funcarg_it): >> > >> ? ? @decorate_it >> > >> ? ? def _test_it(): >> > >> ? ? ? ? # test it >> > >> >> > >> ? ? _test_it() >> > >> >> > >> This works, but it is not nice. I'd rather wrote a decorator like >> > >> >> > >> def decorate_it(f): >> > >> ? ? def _wrap_it(*args, **kwargs): >> > >> ? ? ? ? # wrap f() here >> > >> >> > >> ? ? _wrap_it._varnames = _pytest.core.varnames(f) >> > >> ? ? return _wrap_it >> > >> >> > >> and apply it straight to the test function. >> > >> >> > >> After examining the source code I even expected it to just work >> > >> (magically of course) but it didn't. >> > >> Do you think it is worthwhile? If so I can enter a feature request. >> > >> >> > >> Thanks, >> > >> Vyacheslav >> > >> _______________________________________________ >> > >> py-dev mailing list >> > >> py-dev at codespeak.net >> > >> http://codespeak.net/mailman/listinfo/py-dev >> > > >> > > >> > > > >> _______________________________________________ >> py-dev mailing list >> py-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/py-dev > > From holger at merlinux.eu Wed Jun 1 22:40:01 2011 From: holger at merlinux.eu (holger krekel) Date: Wed, 1 Jun 2011 20:40:01 +0000 Subject: [py-dev] Decorators and funcargs in py.test In-Reply-To: References: <1306785315.3360.1.camel@Klappe2> <1306787910.3360.15.camel@Klappe2> <20110601074913.GE20287@merlinux.eu> Message-ID: <20110601204001.GI20287@merlinux.eu> On Wed, Jun 01, 2011 at 12:31 -0400, Vyacheslav Rafalskiy wrote: > Gentlemen! > > Thanks for your input. I really appreciate it. > > @Ronny > 1. I do agree that creating a new data convention is a big deal. > What I tried to suggest is using something that already exists in > _pytest.core.varnames(). A comment in _pytest.python.getfuncargnames() > indicates that the two functions may merge. If they do, and towards > the former, my original suggestion will *just work*. > > 2. Using decorators is a great deal for me. They are powerful and > expressive. I understand however the reluctance to open a can of worms > here. The replacement using pytest.mark below seems acceptable to me. > It still uses the same decorator function, just at a different place. > > @Holger, Ronny > Here is what I came up with: > > #---------->> in test_1.py > @pytest.mark.timeout(10) > def test_f1(): > # test here > > #---------->> in conftest.py > def pytest_runtest_call(item): > if hasattr(item.obj, 'timeout'): > timeout = item.obj.timeout.args[0] > item.obj = run_with_timeout(timeout)(item.obj) > > Your comments are welcome. it's basically ok but there are some bits that could be improved. You are actually implementing the general test item call. Also I am not sure why you assign "item.obj = ...". I'd probably write something like this: @pytest.mark.tryfirst def pytest_pyfunc_call(pyfuncitem, __multicall__): if 'timeout' in pyfuncitem.keywords: timeout = pyfuncitem.keywords['timeout'].args[0] run_with_timeout(timeout)(__multicall__.execute) main differences: * only applies to python test function calls * hook invocation will be "tried first" before other pytest_pyfunc_call hook impls and it will call those other hooks through the "__multicall__" bit which actually represents the ongoing hook call * will call other hook implementations How do you actually implement run_with_timeout, btw? best, holger From rafalskiy at gmail.com Wed Jun 1 23:16:43 2011 From: rafalskiy at gmail.com (Vyacheslav Rafalskiy) Date: Wed, 1 Jun 2011 17:16:43 -0400 Subject: [py-dev] Decorators and funcargs in py.test In-Reply-To: <20110601204001.GI20287@merlinux.eu> References: <1306785315.3360.1.camel@Klappe2> <1306787910.3360.15.camel@Klappe2> <20110601074913.GE20287@merlinux.eu> <20110601204001.GI20287@merlinux.eu> Message-ID: >> #---------->> ?in test_1.py >> @pytest.mark.timeout(10) >> def test_f1(): >> ? ? # test here >> >> #---------->> ?in conftest.py >> def pytest_runtest_call(item): >> ? ? if hasattr(item.obj, 'timeout'): >> ? ? ? ? timeout = item.obj.timeout.args[0] >> ? ? ? ? item.obj = run_with_timeout(timeout)(item.obj) >> >> Your comments are welcome. > > it's basically ok but there are some bits that could > be improved. ?You are actually implementing the general > test item call. ?Also I am not sure why you assign > "item.obj = ...". I replace (or so I think) the original test function by the decorated one. It gets called elsewhere. > > I'd probably write something like this: > > ? ?@pytest.mark.tryfirst > ? ?def pytest_pyfunc_call(pyfuncitem, __multicall__): > ? ? ? ?if 'timeout' in pyfuncitem.keywords: > ? ? ? ? ? ?timeout = pyfuncitem.keywords['timeout'].args[0] > ? ? ? ? ? ?run_with_timeout(timeout)(__multicall__.execute) this will take a while to digest > main differences: > > * only applies to python test function calls > * hook invocation will be "tried first" before other > ?pytest_pyfunc_call hook impls and it will call those > ?other hooks through the "__multicall__" bit > ?which actually represents the ongoing hook call > * will call other hook implementations > > How do you actually implement run_with_timeout, btw? def run_with_timeout(timeout=None): def _decorator(f): def _wrapper(*args, **kwargs): def _f(*args, **kwargs): try: _result = f(*args, **kwargs) except Exception as e: thread._exception = e else: thread._result = _result thread = threading.Thread(target=_f, args=args, kwargs=kwargs) thread.daemon = True thread._exception = None thread.start() thread.join(timeout=timeout) if thread.isAlive(): raise RuntimeError('function *%s* exceeded configured timeout of %ss' % (f.__name__, timeout)) if thread._exception is None: return thread._result else: raise thread._exception _wrapper.__name__ = f.__name__ return _wrapper return _decorator > > best, > holger > Vyacheslav From holger at merlinux.eu Thu Jun 2 06:46:06 2011 From: holger at merlinux.eu (holger krekel) Date: Thu, 2 Jun 2011 04:46:06 +0000 Subject: [py-dev] Decorators and funcargs in py.test In-Reply-To: References: <1306785315.3360.1.camel@Klappe2> <1306787910.3360.15.camel@Klappe2> <20110601074913.GE20287@merlinux.eu> <20110601204001.GI20287@merlinux.eu> Message-ID: <20110602044606.GJ20287@merlinux.eu> On Wed, Jun 01, 2011 at 17:16 -0400, Vyacheslav Rafalskiy wrote: > >> #---------->> ?in test_1.py > >> @pytest.mark.timeout(10) > >> def test_f1(): > >> ? ? # test here > >> > >> #---------->> ?in conftest.py > >> def pytest_runtest_call(item): > >> ? ? if hasattr(item.obj, 'timeout'): > >> ? ? ? ? timeout = item.obj.timeout.args[0] > >> ? ? ? ? item.obj = run_with_timeout(timeout)(item.obj) > >> > >> Your comments are welcome. > > > > it's basically ok but there are some bits that could > > be improved. ?You are actually implementing the general > > test item call. ?Also I am not sure why you assign > > "item.obj = ...". > > I replace (or so I think) the original test function by the decorated > one. It gets called elsewhere. ah, of course :) > > I'd probably write something like this: > > > > ? ?@pytest.mark.tryfirst > > ? ?def pytest_pyfunc_call(pyfuncitem, __multicall__): > > ? ? ? ?if 'timeout' in pyfuncitem.keywords: > > ? ? ? ? ? ?timeout = pyfuncitem.keywords['timeout'].args[0] > > ? ? ? ? ? ?run_with_timeout(timeout)(__multicall__.execute) > > this will take a while to digest it's actually wrong if run_with_timeout is only decorating but not running the function. I think it makes sense to rather directly call a helper which calls the function (note that __multicall__.execute() will execute the remainder of the hook chain one of which will actually execute the function). Such a helper would look like call_with_timeout(timeout, func) i guess. best, holger > > main differences: > > > > * only applies to python test function calls > > * hook invocation will be "tried first" before other > > ?pytest_pyfunc_call hook impls and it will call those > > ?other hooks through the "__multicall__" bit > > ?which actually represents the ongoing hook call > > * will call other hook implementations > > > > How do you actually implement run_with_timeout, btw? > > def run_with_timeout(timeout=None): > def _decorator(f): > def _wrapper(*args, **kwargs): > def _f(*args, **kwargs): > try: > _result = f(*args, **kwargs) > except Exception as e: > thread._exception = e > else: > thread._result = _result > > thread = threading.Thread(target=_f, args=args, kwargs=kwargs) > thread.daemon = True > thread._exception = None > > thread.start() > thread.join(timeout=timeout) > if thread.isAlive(): > raise RuntimeError('function *%s* exceeded configured > timeout of %ss' % (f.__name__, timeout)) > if thread._exception is None: > return thread._result > else: > raise thread._exception > _wrapper.__name__ = f.__name__ > return _wrapper > return _decorator > > > > > > > best, > > holger > > > > Vyacheslav > From rafalskiy at gmail.com Fri Jun 3 18:06:06 2011 From: rafalskiy at gmail.com (Vyacheslav Rafalskiy) Date: Fri, 3 Jun 2011 12:06:06 -0400 Subject: [py-dev] Decorators and funcargs in py.test In-Reply-To: <20110602044606.GJ20287@merlinux.eu> References: <1306785315.3360.1.camel@Klappe2> <1306787910.3360.15.camel@Klappe2> <20110601074913.GE20287@merlinux.eu> <20110601204001.GI20287@merlinux.eu> <20110602044606.GJ20287@merlinux.eu> Message-ID: On Thu, Jun 2, 2011 at 12:46 AM, holger krekel wrote: > On Wed, Jun 01, 2011 at 17:16 -0400, Vyacheslav Rafalskiy wrote: >> >> #---------->> ?in test_1.py >> >> @pytest.mark.timeout(10) >> >> def test_f1(): >> >> ? ? # test here >> >> >> >> #---------->> ?in conftest.py >> >> def pytest_runtest_call(item): >> >> ? ? if hasattr(item.obj, 'timeout'): >> >> ? ? ? ? timeout = item.obj.timeout.args[0] >> >> ? ? ? ? item.obj = run_with_timeout(timeout)(item.obj) >> >> >> >> Your comments are welcome. >> > >> > it's basically ok but there are some bits that could >> > be improved. ?You are actually implementing the general >> > test item call. ?Also I am not sure why you assign >> > "item.obj = ...". >> >> I replace (or so I think) the original test function by the decorated >> one. It gets called elsewhere. > > ah, of course :) > >> > I'd probably write something like this: >> > >> > ? ?@pytest.mark.tryfirst >> > ? ?def pytest_pyfunc_call(pyfuncitem, __multicall__): >> > ? ? ? ?if 'timeout' in pyfuncitem.keywords: >> > ? ? ? ? ? ?timeout = pyfuncitem.keywords['timeout'].args[0] >> > ? ? ? ? ? ?run_with_timeout(timeout)(__multicall__.execute) >> >> this will take a while to digest > > it's actually wrong if run_with_timeout is only decorating > but not running the function. ?I think it makes sense to > rather directly call a helper which calls the function > (note that __multicall__.execute() will execute the remainder > of the hook chain one of which will actually execute the > function). ?Such a helper would look like > call_with_timeout(timeout, func) i guess. To call the function I need to know what arguments to give it to. The following seems to work, but this is just a guess: def pytest_runtest_call(item): if hasattr(item.obj, 'timeout'): timeout = item.obj.timeout.args[0] run_with_timeout(timeout)(item.obj)(**item.funcargs) I still don't quite get your example, specifically the __multicall__.execute part. Btw, the pytest_pyfunc_call() parameters seem to be in the wrong order based on _pytest.python.py and the prototype in _pytest.hookspec.py only lists one parameter. Vyacheslav From holger at merlinux.eu Fri Jun 3 19:38:44 2011 From: holger at merlinux.eu (holger krekel) Date: Fri, 3 Jun 2011 17:38:44 +0000 Subject: [py-dev] Decorators and funcargs in py.test In-Reply-To: References: <1306785315.3360.1.camel@Klappe2> <1306787910.3360.15.camel@Klappe2> <20110601074913.GE20287@merlinux.eu> <20110601204001.GI20287@merlinux.eu> <20110602044606.GJ20287@merlinux.eu> Message-ID: <20110603173844.GM20287@merlinux.eu> On Fri, Jun 03, 2011 at 12:06 -0400, Vyacheslav Rafalskiy wrote: > On Thu, Jun 2, 2011 at 12:46 AM, holger krekel wrote: > > On Wed, Jun 01, 2011 at 17:16 -0400, Vyacheslav Rafalskiy wrote: > >> >> #---------->> ?in test_1.py > >> >> @pytest.mark.timeout(10) > >> >> def test_f1(): > >> >> ? ? # test here > >> >> > >> >> #---------->> ?in conftest.py > >> >> def pytest_runtest_call(item): > >> >> ? ? if hasattr(item.obj, 'timeout'): > >> >> ? ? ? ? timeout = item.obj.timeout.args[0] > >> >> ? ? ? ? item.obj = run_with_timeout(timeout)(item.obj) > >> >> > >> >> Your comments are welcome. > >> > > >> > it's basically ok but there are some bits that could > >> > be improved. ?You are actually implementing the general > >> > test item call. ?Also I am not sure why you assign > >> > "item.obj = ...". > >> > >> I replace (or so I think) the original test function by the decorated > >> one. It gets called elsewhere. > > > > ah, of course :) > > > >> > I'd probably write something like this: > >> > > >> > ? ?@pytest.mark.tryfirst > >> > ? ?def pytest_pyfunc_call(pyfuncitem, __multicall__): > >> > ? ? ? ?if 'timeout' in pyfuncitem.keywords: > >> > ? ? ? ? ? ?timeout = pyfuncitem.keywords['timeout'].args[0] > >> > ? ? ? ? ? ?run_with_timeout(timeout)(__multicall__.execute) > >> > >> this will take a while to digest > > > > it's actually wrong if run_with_timeout is only decorating > > but not running the function. ?I think it makes sense to > > rather directly call a helper which calls the function > > (note that __multicall__.execute() will execute the remainder > > of the hook chain one of which will actually execute the > > function). ?Such a helper would look like > > call_with_timeout(timeout, func) i guess. > > To call the function I need to know what arguments to give it to. > The following seems to work, but this is just a guess: > > def pytest_runtest_call(item): > if hasattr(item.obj, 'timeout'): > timeout = item.obj.timeout.args[0] > run_with_timeout(timeout)(item.obj)(**item.funcargs) > > I still don't quite get your example, specifically the > __multicall__.execute part. A few things that might help you to understand: * There can be multiple hook functions implementing the same hook. * MultiCall instances manage the calling into a list of hook implementation functions. * A hook impl function can use zero or any number of available arguments. i.e. if a hookspec is "pyest_myfunc(x,y,z)" then a hook impl function can just receive only one of them "pytest_myfunc(x)" This slicing is done from the MultiCall class. * the actual call to a hook impl function is always done by passing keyword arguments (i.e. **kwargs) and thus ordering of parameters is irrelevant. * a hook impl function can receive a "__multicall__" parameter which is its managing class and which it can use to call __multicall__.execute() to execute the rest of the hook implementations. * with all this in mind the few lines of code in _pytest.core.MultiCall hopefully make more sense :) best, holger > Btw, the pytest_pyfunc_call() parameters seem to be in the wrong order > based on _pytest.python.py and the prototype in _pytest.hookspec.py > only lists one parameter. > > > Vyacheslav > From rafalskiy at gmail.com Mon Jun 6 15:24:55 2011 From: rafalskiy at gmail.com (Vyacheslav Rafalskiy) Date: Mon, 6 Jun 2011 09:24:55 -0400 Subject: [py-dev] Decorators and funcargs in py.test In-Reply-To: <20110603173844.GM20287@merlinux.eu> References: <1306785315.3360.1.camel@Klappe2> <1306787910.3360.15.camel@Klappe2> <20110601074913.GE20287@merlinux.eu> <20110601204001.GI20287@merlinux.eu> <20110602044606.GJ20287@merlinux.eu> <20110603173844.GM20287@merlinux.eu> Message-ID: Thanks Holger, It is clearer now. Vyacheslav On Fri, Jun 3, 2011 at 1:38 PM, holger krekel wrote: > On Fri, Jun 03, 2011 at 12:06 -0400, Vyacheslav Rafalskiy wrote: >> On Thu, Jun 2, 2011 at 12:46 AM, holger krekel wrote: >> > On Wed, Jun 01, 2011 at 17:16 -0400, Vyacheslav Rafalskiy wrote: >> >> >> #---------->> ?in test_1.py >> >> >> @pytest.mark.timeout(10) >> >> >> def test_f1(): >> >> >> ? ? # test here >> >> >> >> >> >> #---------->> ?in conftest.py >> >> >> def pytest_runtest_call(item): >> >> >> ? ? if hasattr(item.obj, 'timeout'): >> >> >> ? ? ? ? timeout = item.obj.timeout.args[0] >> >> >> ? ? ? ? item.obj = run_with_timeout(timeout)(item.obj) >> >> >> >> >> >> Your comments are welcome. >> >> > >> >> > it's basically ok but there are some bits that could >> >> > be improved. ?You are actually implementing the general >> >> > test item call. ?Also I am not sure why you assign >> >> > "item.obj = ...". >> >> >> >> I replace (or so I think) the original test function by the decorated >> >> one. It gets called elsewhere. >> > >> > ah, of course :) >> > >> >> > I'd probably write something like this: >> >> > >> >> > ? ?@pytest.mark.tryfirst >> >> > ? ?def pytest_pyfunc_call(pyfuncitem, __multicall__): >> >> > ? ? ? ?if 'timeout' in pyfuncitem.keywords: >> >> > ? ? ? ? ? ?timeout = pyfuncitem.keywords['timeout'].args[0] >> >> > ? ? ? ? ? ?run_with_timeout(timeout)(__multicall__.execute) >> >> >> >> this will take a while to digest >> > >> > it's actually wrong if run_with_timeout is only decorating >> > but not running the function. ?I think it makes sense to >> > rather directly call a helper which calls the function >> > (note that __multicall__.execute() will execute the remainder >> > of the hook chain one of which will actually execute the >> > function). ?Such a helper would look like >> > call_with_timeout(timeout, func) i guess. >> >> To call the function I need to know what arguments to give it to. >> The following seems to work, but this is just a guess: >> >> def pytest_runtest_call(item): >> ? ? if hasattr(item.obj, 'timeout'): >> ? ? ? ? timeout = item.obj.timeout.args[0] >> ? ? ? ? run_with_timeout(timeout)(item.obj)(**item.funcargs) >> >> I still don't quite get your example, specifically the >> __multicall__.execute part. > > A few things that might help you to understand: > > * There can be multiple hook functions implementing the same hook. > * MultiCall instances manage the calling into a list of hook > ?implementation functions. > * A hook impl function can use zero or any number of available > ?arguments. ?i.e. if a hookspec is "pyest_myfunc(x,y,z)" then > ?a hook impl function can just receive only one of them "pytest_myfunc(x)" > ?This slicing is done from the MultiCall class. > * the actual call to a hook impl function is always done by > ?passing keyword arguments (i.e. **kwargs) and thus ordering > ?of parameters is irrelevant. > * a hook impl function can receive a "__multicall__" parameter > ?which is its managing class and which it can use to call > ?__multicall__.execute() to execute the rest of the hook implementations. > * with all this in mind the few lines of code in _pytest.core.MultiCall > ?hopefully make more sense :) > > best, > holger > >> Btw, the pytest_pyfunc_call() parameters seem to be in the wrong order >> based on _pytest.python.py and the prototype in _pytest.hookspec.py >> only lists one parameter. >> >> >> ?Vyacheslav >> > From ralf at systemexit.de Fri Jun 10 17:09:25 2011 From: ralf at systemexit.de (Ralf Schmitt) Date: Fri, 10 Jun 2011 17:09:25 +0200 Subject: [py-dev] error in teardown wrongly reported as error in setup Message-ID: <8739jh910a.fsf@muni.brainbot.com> I have a directory with two test files with the following content: ,----[ test_1.py ] | #! /usr/bin/env py.test | | def pytest_funcarg__hello(request): | def setup(): | return "hello world" | | def teardown(idx): | raise RuntimeError("fail") | | return request.cached_setup( | setup=setup, | teardown=teardown, | scope="module") | | | def test_val(hello): | assert hello == "hello world" `---- ,----[ test_2.py ] | def test_foo(): | pass `---- py.test reports an error for test_foo: ,---- | ========================== test session starts ========================== | platform linux2 -- Python 2.7.2 -- pytest-2.0.3 | collected 2 items | | test_1.py . | test_2.py E | | ================================ ERRORS ================================= | ______________________ ERROR at setup of test_foo _______________________ | | idx = 'hello world' | | def teardown(idx): | > raise RuntimeError("fail") | E RuntimeError: fail | | test_1.py:8: RuntimeError | =================== 1 passed, 1 error in 0.10 seconds =================== `---- Cheers, - Ralf