From holger at merlinux.eu Tue Oct 1 12:39:51 2013 From: holger at merlinux.eu (holger krekel) Date: Tue, 1 Oct 2013 10:39:51 +0000 Subject: [pytest-dev] pytest-2.4: may better fixtures be with you Message-ID: <20131001103951.GV14010@merlinux.eu> The just released pytest-2.4.0 brings many improvements and numerous bug fixes while remaining plugin- and test-suite compatible apart from a few supposedly very minor incompatibilities. See below for a full list of details. A few feature highlights: - new yield-style fixtures `pytest.yield_fixture `_, allowing to use existing with-style context managers in fixture functions. - improved pdb support: ``import pdb ; pdb.set_trace()`` now works without requiring prior disabling of stdout/stderr capturing. Also the ``--pdb`` options works now on collection and internal errors and we introduced a new experimental hook for IDEs/plugins to intercept debugging: ``pytest_exception_interact(node, call, report)``. - shorter monkeypatch variant to allow specifying an import path as a target, for example: ``monkeypatch.setattr("requests.get", myfunc)`` - better unittest/nose compatibility: all teardown methods are now only called if the corresponding setup method succeeded. - integrate tab-completion on command line options if you have `argcomplete `_ configured. - allow boolean expression directly with skipif/xfail if a "reason" is also specified. - a new hook ``pytest_load_initial_conftests`` allows plugins like `pytest-django `_ to influence the environment before conftest files import ``django``. - reporting: color the last line red or green depending if failures/errors occured or everything passed. The documentation has been updated to accomodate the changes, see `http://pytest.org `_ To install or upgrade pytest:: pip install -U pytest # or easy_install -U pytest **Many thanks to all who helped, including Floris Bruynooghe, Brianna Laugher, Andreas Pelme, Anthon van der Neut, Anatoly Bubenkoff, Vladimir Keleshev, Mathieu Agopian, Ronny Pfannschmidt, Christian Theunert and many others.** may passing tests be with you, holger krekel Changes between 2.3.5 and 2.4 ----------------------------------- known incompatibilities: - if calling --genscript from python2.7 or above, you only get a standalone script which works on python2.7 or above. Use Python2.6 to also get a python2.5 compatible version. - all xunit-style teardown methods (nose-style, pytest-style, unittest-style) will not be called if the corresponding setup method failed, see issue322 below. - the pytest_plugin_unregister hook wasn't ever properly called and there is no known implementation of the hook - so it got removed. - pytest.fixture-decorated functions cannot be generators (i.e. use yield) anymore. This change might be reversed in 2.4.1 if it causes unforeseen real-life issues. However, you can always write and return an inner function/generator and change the fixture consumer to iterate over the returned generator. This change was done in lieu of the new ``pytest.yield_fixture`` decorator, see below. new features: - experimentally introduce a new ``pytest.yield_fixture`` decorator which accepts exactly the same parameters as pytest.fixture but mandates a ``yield`` statement instead of a ``return statement`` from fixture functions. This allows direct integration with "with-style" context managers in fixture functions and generally avoids registering of finalization callbacks in favour of treating the "after-yield" as teardown code. Thanks Andreas Pelme, Vladimir Keleshev, Floris Bruynooghe, Ronny Pfannschmidt and many others for discussions. - allow boolean expression directly with skipif/xfail if a "reason" is also specified. Rework skipping documentation to recommend "condition as booleans" because it prevents surprises when importing markers between modules. Specifying conditions as strings will remain fully supported. - reporting: color the last line red or green depending if failures/errors occured or everything passed. thanks Christian Theunert. - make "import pdb ; pdb.set_trace()" work natively wrt capturing (no "-s" needed anymore), making ``pytest.set_trace()`` a mere shortcut. - fix issue181: --pdb now also works on collect errors (and on internal errors) . This was implemented by a slight internal refactoring and the introduction of a new hook ``pytest_exception_interact`` hook (see next item). - fix issue341: introduce new experimental hook for IDEs/terminals to intercept debugging: ``pytest_exception_interact(node, call, report)``. - new monkeypatch.setattr() variant to provide a shorter invocation for patching out classes/functions from modules: monkeypatch.setattr("requests.get", myfunc) will replace the "get" function of the "requests" module with ``myfunc``. - fix issue322: tearDownClass is not run if setUpClass failed. Thanks Mathieu Agopian for the initial fix. Also make all of pytest/nose finalizer mimick the same generic behaviour: if a setupX exists and fails, don't run teardownX. This internally introduces a new method "node.addfinalizer()" helper which can only be called during the setup phase of a node. - simplify pytest.mark.parametrize() signature: allow to pass a CSV-separated string to specify argnames. For example: ``pytest.mark.parametrize("input,expected", [(1,2), (2,3)])`` works as well as the previous: ``pytest.mark.parametrize(("input", "expected"), ...)``. - add support for setUpModule/tearDownModule detection, thanks Brian Okken. - integrate tab-completion on options through use of "argcomplete". Thanks Anthon van der Neut for the PR. - change option names to be hyphen-separated long options but keep the old spelling backward compatible. py.test -h will only show the hyphenated version, for example "--collect-only" but "--collectonly" will remain valid as well (for backward-compat reasons). Many thanks to Anthon van der Neut for the implementation and to Hynek Schlawack for pushing us. - fix issue 308 - allow to mark/xfail/skip individual parameter sets when parametrizing. Thanks Brianna Laugher. - call new experimental pytest_load_initial_conftests hook to allow 3rd party plugins to do something before a conftest is loaded. Bug fixes: - fix issue358 - capturing options are now parsed more properly by using a new parser.parse_known_args method. - pytest now uses argparse instead of optparse (thanks Anthon) which means that "argparse" is added as a dependency if installing into python2.6 environments or below. - fix issue333: fix a case of bad unittest/pytest hook interaction. - PR27: correctly handle nose.SkipTest during collection. Thanks Antonio Cuni, Ronny Pfannschmidt. - fix issue355: junitxml puts name="pytest" attribute to testsuite tag. - fix issue336: autouse fixture in plugins should work again. - fix issue279: improve object comparisons on assertion failure for standard datatypes and recognise collections.abc. Thanks to Brianna Laugher and Mathieu Agopian. - fix issue317: assertion rewriter support for the is_package method - fix issue335: document py.code.ExceptionInfo() object returned from pytest.raises(), thanks Mathieu Agopian. - remove implicit distribute_setup support from setup.py. - fix issue305: ignore any problems when writing pyc files. - SO-17664702: call fixture finalizers even if the fixture function partially failed (finalizers would not always be called before) - fix issue320 - fix class scope for fixtures when mixed with module-level functions. Thanks Anatloy Bubenkoff. - you can specify "-q" or "-qq" to get different levels of "quieter" reporting (thanks Katarzyna Jachim) - fix issue300 - Fix order of conftest loading when starting py.test in a subdirectory. - fix issue323 - sorting of many module-scoped arg parametrizations - make sessionfinish hooks execute with the same cwd-context as at session start (helps fix plugin behaviour which write output files with relative path such as pytest-cov) - fix issue316 - properly reference collection hooks in docs - fix issue 306 - cleanup of -k/-m options to only match markers/test names/keywords respectively. Thanks Wouter van Ackooy. - improved doctest counting for doctests in python modules -- files without any doctest items will not show up anymore and doctest examples are counted as separate test items. thanks Danilo Bellini. - fix issue245 by depending on the released py-1.4.14 which fixes py.io.dupfile to work with files with no mode. Thanks Jason R. Coombs. - fix junitxml generation when test output contains control characters, addressing issue267, thanks Jaap Broekhuizen - fix issue338: honor --tb style for setup/teardown errors as well. Thanks Maho. - fix issue307 - use yaml.safe_load in example, thanks Mark Eichin. - better parametrize error messages, thanks Brianna Laugher - pytest_terminal_summary(terminalreporter) hooks can now use ".section(title)" and ".line(msg)" methods to print extra information at the end of a test run. From vladimir at keleshev.com Tue Oct 1 12:51:30 2013 From: vladimir at keleshev.com (Vladimir Keleshev) Date: Tue, 01 Oct 2013 12:51:30 +0200 Subject: [pytest-dev] please test pytest-2.4.0 release candidate In-Reply-To: <20130930140633.GQ14010@merlinux.eu> References: <20130930140633.GQ14010@merlinux.eu> Message-ID: <776561380624690@web9m.yandex.ru> Hi Holger, Maybe you are still working on the docs, but I just wanted to note that you probably want to remove this paragraph: > ?Some developers also expressed their preference for rather introduce a new @pytest.yieldfixture decorator instead of a keyword argument, or for assuming the above yield-semantics automatically by introspecting if a fixture function is a generator. Depending on more experiences and feedback during the 2.4 cycle, we revisit theses issues. See: http://pytest.org/latest/yieldfixture.html Also, forbidding yield in return-style fixtures was a good decision, I think. ?Vladimir 30.09.2013, 16:06, "holger krekel" : > ?Hi everybody, > > ?could you help to verify that pytest-2.4 will continue to work for your > ?test suite and/or plugins? ?Please report back any issues here or on > ?#pylib freenode. ?Release is scheduled tomorrow (tuesday) noon (GMT) so > ?please hurry :) > > ?The repo is now pretty much in shape for a release and > ?you can install a packaged version like this:: > > ?????pip install -i http://devpi.net/hpk/dev/+simple/ -U pytest > > ?this should give you pytest-2.4.dev13 (see "--version" option). > > ?Below is the long changelog in case you are curious. > > ?Latest two changes: > > ?- the decorator yield_fixture is now there. ?See > ???http://pytest.org/dev/yieldfixture.html > > ?- i did a refactoring of how pytest startup is performed, fixing > ???issue358 and allowing pytest-django and other plugins to > ???perform actions before the first conftest file is loaded. > ???Andreas Pelme already adapted pytest-django to utilize > ???the improvements (mainly you can then use ``--ds=...`` setting > ???and have your conftest.py files import django) and it seems to fix > ???issue359. > > ?Apart from some known incompatibilities (see changelog below) > ?pytest-2.4 aims to be a drop-in replacement for 2.3.5. > > ?cheers, > ?holger > > ?Changes between 2.3.5 and 2.4.DEV > ?----------------------------------- > > ?known incompatibilities: > > ?- if calling --genscript from python2.7 or above, you only get a > ???standalone script which works on python2.7 or above. ?Use Python2.6 > ???to also get a python2.5 compatible version. > > ?- all xunit-style teardown methods (nose-style, pytest-style, > ???unittest-style) will not be called if the corresponding setup method failed, > ???see issue322 below. > > ?- the pytest_plugin_unregister hook wasn't ever properly called > ???and there is no known implementation of the hook - so it got removed. > > ?- pytest.fixture-decorated functions cannot be generators (i.e. use > ???yield) anymore. ?This change might be reversed in 2.4.1 if it causes > ???unforeseen real-life issues. ?However, you can always write and return > ???an inner function/generator and change the fixture consumer to iterate > ???over the returned generator. ?This change was done in lieu of the new > ???``pytest.yield_fixture`` decorator, see below. > > ?new features: > > ?- experimentally introduce a new ``pytest.yield_fixture`` decorator > ???which accepts exactly the same parameters as pytest.fixture but > ???mandates a ``yield`` statement instead of a ``return statement`` from > ???fixture functions. ?This allows direct integration with "with-style" > ???context managers in fixture functions and generally avoids registering > ???of finalization callbacks in favour of treating the "after-yield" as > ???teardown code. ?Thanks Andreas Pelme, Vladimir Keleshev, Floris > ???Bruynooghe, Ronny Pfannschmidt and many others for discussions. > > ?- allow boolean expression directly with skipif/xfail > ???if a "reason" is also specified. ?Rework skipping documentation > ???to recommend "condition as booleans" because it prevents surprises > ???when importing markers between modules. ?Specifying conditions > ???as strings will remain fully supported. > > ?- reporting: color the last line red or green depending if > ???failures/errors occured or everything passed. ?thanks Christian > ???Theunert. > > ?- make "import pdb ; pdb.set_trace()" work natively wrt capturing (no > ???"-s" needed anymore), making ``pytest.set_trace()`` a mere shortcut. > > ?- fix issue181: --pdb now also works on collect errors (and > ???on internal errors) . ?This was implemented by a slight internal > ???refactoring and the introduction of a new hook > ???``pytest_exception_interact`` hook (see next item). > > ?- fix issue341: introduce new experimental hook for IDEs/terminals to > ???intercept debugging: ``pytest_exception_interact(node, call, report)``. > > ?- new monkeypatch.setattr() variant to provide a shorter > ???invocation for patching out classes/functions from modules: > > ??????monkeypatch.setattr("requests.get", myfunc) > > ???will replace the "get" function of the "requests" module with ``myfunc``. > > ?- fix issue322: tearDownClass is not run if setUpClass failed. Thanks > ???Mathieu Agopian for the initial fix. ?Also make all of pytest/nose > ???finalizer mimick the same generic behaviour: if a setupX exists and > ???fails, don't run teardownX. ?This internally introduces a new method > ???"node.addfinalizer()" helper which can only be called during the setup > ???phase of a node. > > ?- simplify pytest.mark.parametrize() signature: allow to pass a > ???CSV-separated string to specify argnames. ?For example: > ???``pytest.mark.parametrize("input,expected", ?[(1,2), (2,3)])`` > ???works as well as the previous: > ???``pytest.mark.parametrize(("input", "expected"), ...)``. > > ?- add support for setUpModule/tearDownModule detection, thanks Brian Okken. > > ?- integrate tab-completion on options through use of "argcomplete". > ???Thanks Anthon van der Neut for the PR. > > ?- change option names to be hyphen-separated long options but keep the > ???old spelling backward compatible. ?py.test -h will only show the > ???hyphenated version, for example "--collect-only" but "--collectonly" > ???will remain valid as well (for backward-compat reasons). ?Many thanks to > ???Anthon van der Neut for the implementation and to Hynek Schlawack for > ???pushing us. > > ?- fix issue 308 - allow to mark/xfail/skip individual parameter sets > ???when parametrizing. ?Thanks Brianna Laugher. > > ?- call new experimental pytest_load_initial_conftests hook to allow > ???3rd party plugins to do something before a conftest is loaded. > > ?Bug fixes: > > ?- fix issue358 - capturing options are now parsed more properly > ???by using a new parser.parse_known_args method. > > ?- pytest now uses argparse instead of optparse (thanks Anthon) which > ???means that "argparse" is added as a dependency if installing into python2.6 > ???environments or below. > > ?- fix issue333: fix a case of bad unittest/pytest hook interaction. > > ?- PR27: correctly handle nose.SkipTest during collection. ?Thanks > ???Antonio Cuni, Ronny Pfannschmidt. > > ?- fix issue355: junitxml puts name="pytest" attribute to testsuite tag. > > ?- fix issue336: autouse fixture in plugins should work again. > > ?- fix issue279: improve object comparisons on assertion failure > ???for standard datatypes and recognise collections.abc. ?Thanks to > ???Brianna Laugher and Mathieu Agopian. > > ?- fix issue317: assertion rewriter support for the is_package method > > ?- fix issue335: document py.code.ExceptionInfo() object returned > ???from pytest.raises(), thanks Mathieu Agopian. > > ?- remove implicit distribute_setup support from setup.py. > > ?- fix issue305: ignore any problems when writing pyc files. > > ?- SO-17664702: call fixture finalizers even if the fixture function > ???partially failed (finalizers would not always be called before) > > ?- fix issue320 - fix class scope for fixtures when mixed with > ???module-level functions. ?Thanks Anatloy Bubenkoff. > > ?- you can specify "-q" or "-qq" to get different levels of "quieter" > ???reporting (thanks Katarzyna Jachim) > > ?- fix issue300 - Fix order of conftest loading when starting py.test > ???in a subdirectory. > > ?- fix issue323 - sorting of many module-scoped arg parametrizations > > ?- make sessionfinish hooks execute with the same cwd-context as at > ???session start (helps fix plugin behaviour which write output files > ???with relative path such as pytest-cov) > > ?- fix issue316 - properly reference collection hooks in docs > > ?- fix issue 306 - cleanup of -k/-m options to only match markers/test > ???names/keywords respectively. ?Thanks Wouter van Ackooy. > > ?- improved doctest counting for doctests in python modules -- > ???files without any doctest items will not show up anymore > ???and doctest examples are counted as separate test items. > ???thanks Danilo Bellini. > > ?- fix issue245 by depending on the released py-1.4.14 > ???which fixes py.io.dupfile to work with files with no > ???mode. Thanks Jason R. Coombs. > > ?- fix junitxml generation when test output contains control characters, > ???addressing issue267, thanks Jaap Broekhuizen > > ?- fix issue338: honor --tb style for setup/teardown errors as well. ?Thanks Maho. > > ?- fix issue307 - use yaml.safe_load in example, thanks Mark Eichin. > > ?- better parametrize error messages, thanks Brianna Laugher > > ?- pytest_terminal_summary(terminalreporter) hooks can now use > ???".section(title)" and ".line(msg)" methods to print extra > ???information at the end of a test run. > ?_______________________________________________ > ?Pytest-dev mailing list > ?Pytest-dev at python.org > ?https://mail.python.org/mailman/listinfo/pytest-dev From holger at merlinux.eu Tue Oct 1 13:11:24 2013 From: holger at merlinux.eu (holger krekel) Date: Tue, 1 Oct 2013 11:11:24 +0000 Subject: [pytest-dev] please test pytest-2.4.0 release candidate In-Reply-To: <776561380624690@web9m.yandex.ru> References: <20130930140633.GQ14010@merlinux.eu> <776561380624690@web9m.yandex.ru> Message-ID: <20131001111124.GW14010@merlinux.eu> hi Vladimir, it had already released but just re-pushed the docs striking the para. cheers and thanks for your contribution, holger On Tue, Oct 01, 2013 at 12:51 +0200, Vladimir Keleshev wrote: > Hi Holger, > > Maybe you are still working on the docs, but I just wanted to note that you probably want to remove this paragraph: > > > ?Some developers also expressed their preference for rather introduce a new @pytest.yieldfixture decorator instead of a keyword argument, or for assuming the above yield-semantics automatically by introspecting if a fixture function is a generator. Depending on more experiences and feedback during the 2.4 cycle, we revisit theses issues. > > See: http://pytest.org/latest/yieldfixture.html > > Also, forbidding yield in return-style fixtures was a good decision, I think. > > ?Vladimir > > > 30.09.2013, 16:06, "holger krekel" : > > > ?Hi everybody, > > > > ?could you help to verify that pytest-2.4 will continue to work for your > > ?test suite and/or plugins? ?Please report back any issues here or on > > ?#pylib freenode. ?Release is scheduled tomorrow (tuesday) noon (GMT) so > > ?please hurry :) > > > > ?The repo is now pretty much in shape for a release and > > ?you can install a packaged version like this:: > > > > ?????pip install -i http://devpi.net/hpk/dev/+simple/ -U pytest > > > > ?this should give you pytest-2.4.dev13 (see "--version" option). > > > > ?Below is the long changelog in case you are curious. > > > > ?Latest two changes: > > > > ?- the decorator yield_fixture is now there. ?See > > ???http://pytest.org/dev/yieldfixture.html > > > > ?- i did a refactoring of how pytest startup is performed, fixing > > ???issue358 and allowing pytest-django and other plugins to > > ???perform actions before the first conftest file is loaded. > > ???Andreas Pelme already adapted pytest-django to utilize > > ???the improvements (mainly you can then use ``--ds=...`` setting > > ???and have your conftest.py files import django) and it seems to fix > > ???issue359. > > > > ?Apart from some known incompatibilities (see changelog below) > > ?pytest-2.4 aims to be a drop-in replacement for 2.3.5. > > > > ?cheers, > > ?holger > > > > ?Changes between 2.3.5 and 2.4.DEV > > ?----------------------------------- > > > > ?known incompatibilities: > > > > ?- if calling --genscript from python2.7 or above, you only get a > > ???standalone script which works on python2.7 or above. ?Use Python2.6 > > ???to also get a python2.5 compatible version. > > > > ?- all xunit-style teardown methods (nose-style, pytest-style, > > ???unittest-style) will not be called if the corresponding setup method failed, > > ???see issue322 below. > > > > ?- the pytest_plugin_unregister hook wasn't ever properly called > > ???and there is no known implementation of the hook - so it got removed. > > > > ?- pytest.fixture-decorated functions cannot be generators (i.e. use > > ???yield) anymore. ?This change might be reversed in 2.4.1 if it causes > > ???unforeseen real-life issues. ?However, you can always write and return > > ???an inner function/generator and change the fixture consumer to iterate > > ???over the returned generator. ?This change was done in lieu of the new > > ???``pytest.yield_fixture`` decorator, see below. > > > > ?new features: > > > > ?- experimentally introduce a new ``pytest.yield_fixture`` decorator > > ???which accepts exactly the same parameters as pytest.fixture but > > ???mandates a ``yield`` statement instead of a ``return statement`` from > > ???fixture functions. ?This allows direct integration with "with-style" > > ???context managers in fixture functions and generally avoids registering > > ???of finalization callbacks in favour of treating the "after-yield" as > > ???teardown code. ?Thanks Andreas Pelme, Vladimir Keleshev, Floris > > ???Bruynooghe, Ronny Pfannschmidt and many others for discussions. > > > > ?- allow boolean expression directly with skipif/xfail > > ???if a "reason" is also specified. ?Rework skipping documentation > > ???to recommend "condition as booleans" because it prevents surprises > > ???when importing markers between modules. ?Specifying conditions > > ???as strings will remain fully supported. > > > > ?- reporting: color the last line red or green depending if > > ???failures/errors occured or everything passed. ?thanks Christian > > ???Theunert. > > > > ?- make "import pdb ; pdb.set_trace()" work natively wrt capturing (no > > ???"-s" needed anymore), making ``pytest.set_trace()`` a mere shortcut. > > > > ?- fix issue181: --pdb now also works on collect errors (and > > ???on internal errors) . ?This was implemented by a slight internal > > ???refactoring and the introduction of a new hook > > ???``pytest_exception_interact`` hook (see next item). > > > > ?- fix issue341: introduce new experimental hook for IDEs/terminals to > > ???intercept debugging: ``pytest_exception_interact(node, call, report)``. > > > > ?- new monkeypatch.setattr() variant to provide a shorter > > ???invocation for patching out classes/functions from modules: > > > > ??????monkeypatch.setattr("requests.get", myfunc) > > > > ???will replace the "get" function of the "requests" module with ``myfunc``. > > > > ?- fix issue322: tearDownClass is not run if setUpClass failed. Thanks > > ???Mathieu Agopian for the initial fix. ?Also make all of pytest/nose > > ???finalizer mimick the same generic behaviour: if a setupX exists and > > ???fails, don't run teardownX. ?This internally introduces a new method > > ???"node.addfinalizer()" helper which can only be called during the setup > > ???phase of a node. > > > > ?- simplify pytest.mark.parametrize() signature: allow to pass a > > ???CSV-separated string to specify argnames. ?For example: > > ???``pytest.mark.parametrize("input,expected", ?[(1,2), (2,3)])`` > > ???works as well as the previous: > > ???``pytest.mark.parametrize(("input", "expected"), ...)``. > > > > ?- add support for setUpModule/tearDownModule detection, thanks Brian Okken. > > > > ?- integrate tab-completion on options through use of "argcomplete". > > ???Thanks Anthon van der Neut for the PR. > > > > ?- change option names to be hyphen-separated long options but keep the > > ???old spelling backward compatible. ?py.test -h will only show the > > ???hyphenated version, for example "--collect-only" but "--collectonly" > > ???will remain valid as well (for backward-compat reasons). ?Many thanks to > > ???Anthon van der Neut for the implementation and to Hynek Schlawack for > > ???pushing us. > > > > ?- fix issue 308 - allow to mark/xfail/skip individual parameter sets > > ???when parametrizing. ?Thanks Brianna Laugher. > > > > ?- call new experimental pytest_load_initial_conftests hook to allow > > ???3rd party plugins to do something before a conftest is loaded. > > > > ?Bug fixes: > > > > ?- fix issue358 - capturing options are now parsed more properly > > ???by using a new parser.parse_known_args method. > > > > ?- pytest now uses argparse instead of optparse (thanks Anthon) which > > ???means that "argparse" is added as a dependency if installing into python2.6 > > ???environments or below. > > > > ?- fix issue333: fix a case of bad unittest/pytest hook interaction. > > > > ?- PR27: correctly handle nose.SkipTest during collection. ?Thanks > > ???Antonio Cuni, Ronny Pfannschmidt. > > > > ?- fix issue355: junitxml puts name="pytest" attribute to testsuite tag. > > > > ?- fix issue336: autouse fixture in plugins should work again. > > > > ?- fix issue279: improve object comparisons on assertion failure > > ???for standard datatypes and recognise collections.abc. ?Thanks to > > ???Brianna Laugher and Mathieu Agopian. > > > > ?- fix issue317: assertion rewriter support for the is_package method > > > > ?- fix issue335: document py.code.ExceptionInfo() object returned > > ???from pytest.raises(), thanks Mathieu Agopian. > > > > ?- remove implicit distribute_setup support from setup.py. > > > > ?- fix issue305: ignore any problems when writing pyc files. > > > > ?- SO-17664702: call fixture finalizers even if the fixture function > > ???partially failed (finalizers would not always be called before) > > > > ?- fix issue320 - fix class scope for fixtures when mixed with > > ???module-level functions. ?Thanks Anatloy Bubenkoff. > > > > ?- you can specify "-q" or "-qq" to get different levels of "quieter" > > ???reporting (thanks Katarzyna Jachim) > > > > ?- fix issue300 - Fix order of conftest loading when starting py.test > > ???in a subdirectory. > > > > ?- fix issue323 - sorting of many module-scoped arg parametrizations > > > > ?- make sessionfinish hooks execute with the same cwd-context as at > > ???session start (helps fix plugin behaviour which write output files > > ???with relative path such as pytest-cov) > > > > ?- fix issue316 - properly reference collection hooks in docs > > > > ?- fix issue 306 - cleanup of -k/-m options to only match markers/test > > ???names/keywords respectively. ?Thanks Wouter van Ackooy. > > > > ?- improved doctest counting for doctests in python modules -- > > ???files without any doctest items will not show up anymore > > ???and doctest examples are counted as separate test items. > > ???thanks Danilo Bellini. > > > > ?- fix issue245 by depending on the released py-1.4.14 > > ???which fixes py.io.dupfile to work with files with no > > ???mode. Thanks Jason R. Coombs. > > > > ?- fix junitxml generation when test output contains control characters, > > ???addressing issue267, thanks Jaap Broekhuizen > > > > ?- fix issue338: honor --tb style for setup/teardown errors as well. ?Thanks Maho. > > > > ?- fix issue307 - use yaml.safe_load in example, thanks Mark Eichin. > > > > ?- better parametrize error messages, thanks Brianna Laugher > > > > ?- pytest_terminal_summary(terminalreporter) hooks can now use > > ???".section(title)" and ".line(msg)" methods to print extra > > ???information at the end of a test run. > > ?_______________________________________________ > > ?Pytest-dev mailing list > > ?Pytest-dev at python.org > > ?https://mail.python.org/mailman/listinfo/pytest-dev > From faassen at startifact.com Tue Oct 1 13:42:30 2013 From: faassen at startifact.com (Martijn Faassen) Date: Tue, 1 Oct 2013 13:42:30 +0200 Subject: [pytest-dev] please test pytest-2.4.0 release candidate In-Reply-To: <20131001111124.GW14010@merlinux.eu> References: <20130930140633.GQ14010@merlinux.eu> <776561380624690@web9m.yandex.ru> <20131001111124.GW14010@merlinux.eu> Message-ID: Hey, Just a few language issues while scanning through it to see what's up: http://pytest.org/latest/yieldfixture.html seemlessly -> seamlessly "usually yield is typically used" -> a bit double, get rid of the usually or the typically. "Some developers also expressed their preference for rather introduce a new @pytest.yieldfixture decorator instead of a keyword argument" -> "Some developers expressed their preference for a new @pytest.yieldfixture decorator instead of using keyword arguments" reads better to me. But isn't this exactly what the spelling is now? This seems to be out of date. Regards, Martijn From holger at merlinux.eu Tue Oct 1 14:31:09 2013 From: holger at merlinux.eu (holger krekel) Date: Tue, 1 Oct 2013 12:31:09 +0000 Subject: [pytest-dev] please test pytest-2.4.0 release candidate In-Reply-To: References: <20130930140633.GQ14010@merlinux.eu> <776561380624690@web9m.yandex.ru> <20131001111124.GW14010@merlinux.eu> Message-ID: <20131001123108.GY14010@merlinux.eu> fixed, thanks Martijn, holger On Tue, Oct 01, 2013 at 13:42 +0200, Martijn Faassen wrote: > Hey, > > Just a few language issues while scanning through it to see what's up: > > http://pytest.org/latest/yieldfixture.html > > seemlessly -> seamlessly > > "usually yield is typically used" -> a bit double, get rid of the > usually or the typically. > > "Some developers also expressed their preference for rather introduce > a new @pytest.yieldfixture decorator instead of a keyword argument" -> > > "Some developers expressed their preference for a new > @pytest.yieldfixture decorator instead of using keyword arguments" > reads better to me. > > But isn't this exactly what the spelling is now? This seems to be out of date. > > Regards, > > Martijn > _______________________________________________ > Pytest-dev mailing list > Pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > From holger at merlinux.eu Wed Oct 2 09:21:08 2013 From: holger at merlinux.eu (holger krekel) Date: Wed, 2 Oct 2013 07:21:08 +0000 Subject: [pytest-dev] pytest-2.4.1: fix three regressions (of 2.4 compared to 2.3.5) In-Reply-To: <20131001103951.GV14010@merlinux.eu> References: <20131001103951.GV14010@merlinux.eu> Message-ID: <20131002072108.GZ14010@merlinux.eu> pytest-2.4.1 is a quick follow up release to fix three regressions compared to 2.3.5 before they hit more people: - When using parser.addoption() unicode arguments to the "type" keyword should also be converted to the respective types. thanks Floris Bruynooghe, @dnozay. (fixes issue360 and issue362) - fix dotted filename completion when using argcomplete thanks Anthon van der Neuth. (fixes issue361) - fix regression when a 1-tuple ("arg",) is used for specifying parametrization (the values of the parametrization were passed nested in a tuple). Thanks Donald Stufft. - also merge doc typo fixes, thanks Andy Dirnberger as usual, docs at http://pytest.org and upgrades via:: pip install -U pytest have fun and keep bug reports coming! holger krekel From holger at merlinux.eu Fri Oct 4 14:41:32 2013 From: holger at merlinux.eu (holger krekel) Date: Fri, 4 Oct 2013 12:41:32 +0000 Subject: [pytest-dev] pytest-2.4.2: colorama on win32, plugin/tmpdir/other fixes In-Reply-To: <20131002072108.GZ14010@merlinux.eu> References: <20131001103951.GV14010@merlinux.eu> <20131002072108.GZ14010@merlinux.eu> Message-ID: <20131004124132.GD14010@merlinux.eu> pytest-2.4.2 is another bug-fixing release of the 2.4 series bringing another few fixes and little improvements: - on Windows require colorama and a newer py lib so that py.io.TerminalWriter() now uses colorama instead of its own ctypes hacks. (fixes issue365) thanks Paul Moore for bringing it up. - fix "-k" matching of tests where "repr" and "attr" and other names would cause wrong matches because of an internal implementation quirk (don't ask) which is now properly implemented. fixes issue345. - avoid tmpdir fixture to create too long filenames especially when parametrization is used (issue354) - fix pytest-pep8 and pytest-flakes / pytest interactions (collection names in mark plugin was assuming an item always has a function which is not true for those plugins etc.) Thanks Andi Zeidler. - introduce node.get_marker/node.add_marker API for plugins like pytest-pep8 and pytest-flakes to avoid the messy details of the node.keywords pseudo-dicts. Adapated docs. - remove attempt to "dup" stdout at startup as it's icky. the normal capturing should catch enough possibilities of tests messing up standard FDs. - add pluginmanager.do_configure(config) as a link to config.do_configure() for plugin-compatibility as usual, docs at http://pytest.org and upgrades via:: pip install -U pytest have fun, holger krekel From holger at merlinux.eu Fri Oct 4 15:09:33 2013 From: holger at merlinux.eu (holger krekel) Date: Fri, 4 Oct 2013 13:09:33 +0000 Subject: [pytest-dev] (new) pytest plugins / note to plugin authors Message-ID: <20131004130933.GE14010@merlinux.eu> I just did three pytest plugin releases to get out a few minor fixes and improvements. see below. As far as i am aware most existing plugins (as well as the new releases here) should be compatible to pytest-2.4.2. Some plugins like pytest-capturelog have seen their latest release three years ago but remain functional. A good sign that the plugin system is reasonable i guess. If someone wants to help with organizing plugin compatibility tests and maintaining a status page, i'd really appreciate that. With over 50 existing plugins it's not easy to stay on top. The current pytest.org page is only a start: http://pytest.org/latest/plugins.html#extplugins and here is the full pypi list: https://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search there are also some github/bitbucket-only plugins, in addition. And finally a note to plugin authors: please be subscribed to pytest-dev and post new plugin releases here. best, holger pytest-xdist-1.9: - changed LICENSE to MIT - fix duplicate reported test ids with --looponfailing (thanks Jeremy Thurgood) - fix pytest issue41: re-run tests on all file changes, not just randomly select ones like .py/.c. - fix pytest issue347: slaves running on top of Python3.2 will set PYTHONDONTWRITEYBTECODE to 1 to avoid import concurrency bugs. pytest-xprocess-0.8: - support python3 (tested on linux/win32) - split out pytest independent process support into xprocess.py - add info.kill() method and some smaller refactoring - fix various windows related Popen / killing details - add tests pytest-pep8: - use pytest-2.4.2 node.add_marker() API for adding "pep8" marker From nicoddemus at gmail.com Fri Oct 4 17:43:53 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Fri, 4 Oct 2013 12:43:53 -0300 Subject: [pytest-dev] (new) pytest plugins / note to plugin authors In-Reply-To: <20131004130933.GE14010@merlinux.eu> References: <20131004130933.GE14010@merlinux.eu> Message-ID: Hi Holger, I would be glad to help on that front, if I can. :) Did you give any thoughts on how this could be done? Cheers, On Fri, Oct 4, 2013 at 10:09 AM, holger krekel wrote: > > I just did three pytest plugin releases to get out a few minor > fixes and improvements. see below. As far as i am aware most > existing plugins (as well as the new releases here) should be > compatible to pytest-2.4.2. Some plugins like pytest-capturelog > have seen their latest release three years ago but remain functional. > A good sign that the plugin system is reasonable i guess. > > If someone wants to help with organizing plugin compatibility > tests and maintaining a status page, i'd really appreciate that. > With over 50 existing plugins it's not easy to stay on top. > The current pytest.org page is only a start: > > http://pytest.org/latest/plugins.html#extplugins > > and here is the full pypi list: > > > https://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search > > there are also some github/bitbucket-only plugins, in addition. > > And finally a note to plugin authors: please be subscribed > to pytest-dev and post new plugin releases here. > > best, > holger > > pytest-xdist-1.9: > > - changed LICENSE to MIT > > - fix duplicate reported test ids with --looponfailing > (thanks Jeremy Thurgood) > > - fix pytest issue41: re-run tests on all file changes, not just > randomly select ones like .py/.c. > > - fix pytest issue347: slaves running on top of Python3.2 > will set PYTHONDONTWRITEYBTECODE to 1 to avoid import concurrency > bugs. > > pytest-xprocess-0.8: > > - support python3 (tested on linux/win32) > > - split out pytest independent process support into xprocess.py > > - add info.kill() method and some smaller refactoring > > - fix various windows related Popen / killing details > > - add tests > > > pytest-pep8: > > - use pytest-2.4.2 node.add_marker() API for adding "pep8" marker > > _______________________________________________ > Pytest-dev mailing list > Pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Fri Oct 4 21:45:31 2013 From: holger at merlinux.eu (holger krekel) Date: Fri, 4 Oct 2013 19:45:31 +0000 Subject: [pytest-dev] (new) pytest plugins / note to plugin authors In-Reply-To: References: <20131004130933.GE14010@merlinux.eu> Message-ID: <20131004194531.GH14010@merlinux.eu> Hi Bruno, On Fri, Oct 04, 2013 at 12:43 -0300, Bruno Oliveira wrote: > Hi Holger, > > I would be glad to help on that front, if I can. :) great to hear! > Did you give any thoughts on how this could be done? note sure in detail but i guess each plugin could be presented with this information: pluginname and latest version, pypi link, maintainer/author, repository link, one-line description, test status py27/py33 and some way to re-generate this information (especially the test status). I guess we could start with considering plugins that have a tox.ini and then write a script to download the plugin, unpack it, run tests etc. The other part is how to present this info, but e.g. a CSV file can be rendered by sphinx and the main docs could include it or we make a separate page if need be. thanks for considering to help! holger > Cheers, > > > > > > > On Fri, Oct 4, 2013 at 10:09 AM, holger krekel wrote: > > > > > I just did three pytest plugin releases to get out a few minor > > fixes and improvements. see below. As far as i am aware most > > existing plugins (as well as the new releases here) should be > > compatible to pytest-2.4.2. Some plugins like pytest-capturelog > > have seen their latest release three years ago but remain functional. > > A good sign that the plugin system is reasonable i guess. > > > > If someone wants to help with organizing plugin compatibility > > tests and maintaining a status page, i'd really appreciate that. > > With over 50 existing plugins it's not easy to stay on top. > > The current pytest.org page is only a start: > > > > http://pytest.org/latest/plugins.html#extplugins > > > > and here is the full pypi list: > > > > > > https://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search > > > > there are also some github/bitbucket-only plugins, in addition. > > > > And finally a note to plugin authors: please be subscribed > > to pytest-dev and post new plugin releases here. > > > > best, > > holger > > > > pytest-xdist-1.9: > > > > - changed LICENSE to MIT > > > > - fix duplicate reported test ids with --looponfailing > > (thanks Jeremy Thurgood) > > > > - fix pytest issue41: re-run tests on all file changes, not just > > randomly select ones like .py/.c. > > > > - fix pytest issue347: slaves running on top of Python3.2 > > will set PYTHONDONTWRITEYBTECODE to 1 to avoid import concurrency > > bugs. > > > > pytest-xprocess-0.8: > > > > - support python3 (tested on linux/win32) > > > > - split out pytest independent process support into xprocess.py > > > > - add info.kill() method and some smaller refactoring > > > > - fix various windows related Popen / killing details > > > > - add tests > > > > > > pytest-pep8: > > > > - use pytest-2.4.2 node.add_marker() API for adding "pep8" marker > > > > _______________________________________________ > > Pytest-dev mailing list > > Pytest-dev at python.org > > https://mail.python.org/mailman/listinfo/pytest-dev > > From nicoddemus at gmail.com Fri Oct 4 23:06:36 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Fri, 4 Oct 2013 18:06:36 -0300 Subject: [pytest-dev] (new) pytest plugins / note to plugin authors In-Reply-To: <20131004194531.GH14010@merlinux.eu> References: <20131004130933.GE14010@merlinux.eu> <20131004194531.GH14010@merlinux.eu> Message-ID: Hi Holger, On Fri, Oct 4, 2013 at 4:45 PM, holger krekel wrote: > Hi Bruno, > > On Fri, Oct 04, 2013 at 12:43 -0300, Bruno Oliveira wrote: > > Hi Holger, > > Did you give any thoughts on how this could be done? > > note sure in detail but i guess each plugin could be presented > with this information: > > pluginname and latest version, pypi link, maintainer/author, repository > link, > one-line description, test status py27/py33 > Are you interested in displaying compatibility to several py.test versions, or only to the latest one? and some way to re-generate this information (especially the test status). > I guess we could start with considering plugins that have a tox.ini > and then write a script to download the plugin, unpack it, run tests etc. > I considered this, but then I realize we might have to implement several features to support that, like installing dependencies: pytest-qt for instance requires PySide, which must be installed for it to run its own tests. Other plugins might have other requirements to run its tests (like having a browser open for headless testing), which would have to be supported as well and so on. I fear we would have to implement a lot of infrastructure similar to what travis-ci.org or d rone.iodoes for example. Perhaps we could adopt a solution similar to coveralls.io, in which the author is responsible to upload coverage data, and the site collects and displays that information. We could provide a script or code snippet to be included by the plugin author in he's C.I of choice that would generate and upload pytest-compatibility information. This could be easily hosted in a PaaS like Heroku. We would have to count on plugin authors' contribution to the system for it to work on this scheme, of course. The other part is how to present this info, but e.g. a CSV file > can be rendered by sphinx and the main docs could include it > or we make a separate page if need be. > thanks for considering to help! > Glad to. :) Best Regards, Bruno > > holger > > > Cheers, > > > > > > > > > > > > > > On Fri, Oct 4, 2013 at 10:09 AM, holger krekel > wrote: > > > > > > > > I just did three pytest plugin releases to get out a few minor > > > fixes and improvements. see below. As far as i am aware most > > > existing plugins (as well as the new releases here) should be > > > compatible to pytest-2.4.2. Some plugins like pytest-capturelog > > > have seen their latest release three years ago but remain functional. > > > A good sign that the plugin system is reasonable i guess. > > > > > > If someone wants to help with organizing plugin compatibility > > > tests and maintaining a status page, i'd really appreciate that. > > > With over 50 existing plugins it's not easy to stay on top. > > > The current pytest.org page is only a start: > > > > > > http://pytest.org/latest/plugins.html#extplugins > > > > > > and here is the full pypi list: > > > > > > > > > > https://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search > > > > > > there are also some github/bitbucket-only plugins, in addition. > > > > > > And finally a note to plugin authors: please be subscribed > > > to pytest-dev and post new plugin releases here. > > > > > > best, > > > holger > > > > > > pytest-xdist-1.9: > > > > > > - changed LICENSE to MIT > > > > > > - fix duplicate reported test ids with --looponfailing > > > (thanks Jeremy Thurgood) > > > > > > - fix pytest issue41: re-run tests on all file changes, not just > > > randomly select ones like .py/.c. > > > > > > - fix pytest issue347: slaves running on top of Python3.2 > > > will set PYTHONDONTWRITEYBTECODE to 1 to avoid import concurrency > > > bugs. > > > > > > pytest-xprocess-0.8: > > > > > > - support python3 (tested on linux/win32) > > > > > > - split out pytest independent process support into xprocess.py > > > > > > - add info.kill() method and some smaller refactoring > > > > > > - fix various windows related Popen / killing details > > > > > > - add tests > > > > > > > > > pytest-pep8: > > > > > > - use pytest-2.4.2 node.add_marker() API for adding "pep8" marker > > > > > > _______________________________________________ > > > Pytest-dev mailing list > > > Pytest-dev at python.org > > > https://mail.python.org/mailman/listinfo/pytest-dev > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Fri Oct 4 23:25:20 2013 From: holger at merlinux.eu (holger krekel) Date: Fri, 4 Oct 2013 21:25:20 +0000 Subject: [pytest-dev] (new) pytest plugins / note to plugin authors In-Reply-To: References: <20131004130933.GE14010@merlinux.eu> <20131004194531.GH14010@merlinux.eu> Message-ID: <20131004212520.GJ14010@merlinux.eu> Hi Bruno, On Fri, Oct 04, 2013 at 18:06 -0300, Bruno Oliveira wrote: > Hi Holger, > > On Fri, Oct 4, 2013 at 4:45 PM, holger krekel wrote: > > > Hi Bruno, > > > > On Fri, Oct 04, 2013 at 12:43 -0300, Bruno Oliveira wrote: > > > Hi Holger, > > > Did you give any thoughts on how this could be done? > > > > note sure in detail but i guess each plugin could be presented > > with this information: > > > > pluginname and latest version, pypi link, maintainer/author, repository > > link, > > one-line description, test status py27/py33 > > > > Are you interested in displaying compatibility to several py.test versions, > or only to the latest one? ideally latest release and trunk ... > and some way to re-generate this information (especially the test status). > > I guess we could start with considering plugins that have a tox.ini > > and then write a script to download the plugin, unpack it, run tests etc. > > > > I considered this, but then I realize we might have to implement several > features to support that, like installing dependencies: pytest-qt for > instance requires PySide, which must be installed for it to run its own > tests. Other plugins might have other requirements to run its tests (like > having a browser open for headless testing), which would have to be > supported as well and so on. I fear we would have to implement a lot of > infrastructure similar to what travis-ci.org or d > rone.iodoes for example. > > Perhaps we could adopt a solution similar to coveralls.io, in which the > author is responsible to upload coverage data, and the site collects and > displays that information. We could provide a script or code snippet to be > included by the plugin author in he's C.I of choice that would generate and > upload pytest-compatibility information. This could be easily hosted in a > PaaS like Heroku. > > We would have to count on plugin authors' contribution to the system for it > to work on this scheme, of course. I wonder how difficult most plugins really are to setup. PyQT might be difficult but OTOH many others should just be fine. And pushing the responsibility for performing tests and uploading results still would not get the benefit of testing current plugin releases against a new core pytest release (which can break many plugins whereas a new plugin release can just break that plugin :). So i'd think it's best to check how many of the plugins even provide tests. But even just collecting the other information i mentioned (without the test status / running questions) would already be helpful, btw. cheers, holger > The other part is how to present this info, but e.g. a CSV file > > can be rendered by sphinx and the main docs could include it > > or we make a separate page if need be. > > > > thanks for considering to help! > > > > Glad to. :) > > Best Regards, > Bruno > > > > > holger > > > > > Cheers, > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 4, 2013 at 10:09 AM, holger krekel > > wrote: > > > > > > > > > > > I just did three pytest plugin releases to get out a few minor > > > > fixes and improvements. see below. As far as i am aware most > > > > existing plugins (as well as the new releases here) should be > > > > compatible to pytest-2.4.2. Some plugins like pytest-capturelog > > > > have seen their latest release three years ago but remain functional. > > > > A good sign that the plugin system is reasonable i guess. > > > > > > > > If someone wants to help with organizing plugin compatibility > > > > tests and maintaining a status page, i'd really appreciate that. > > > > With over 50 existing plugins it's not easy to stay on top. > > > > The current pytest.org page is only a start: > > > > > > > > http://pytest.org/latest/plugins.html#extplugins > > > > > > > > and here is the full pypi list: > > > > > > > > > > > > > > https://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search > > > > > > > > there are also some github/bitbucket-only plugins, in addition. > > > > > > > > And finally a note to plugin authors: please be subscribed > > > > to pytest-dev and post new plugin releases here. > > > > > > > > best, > > > > holger > > > > > > > > pytest-xdist-1.9: > > > > > > > > - changed LICENSE to MIT > > > > > > > > - fix duplicate reported test ids with --looponfailing > > > > (thanks Jeremy Thurgood) > > > > > > > > - fix pytest issue41: re-run tests on all file changes, not just > > > > randomly select ones like .py/.c. > > > > > > > > - fix pytest issue347: slaves running on top of Python3.2 > > > > will set PYTHONDONTWRITEYBTECODE to 1 to avoid import concurrency > > > > bugs. > > > > > > > > pytest-xprocess-0.8: > > > > > > > > - support python3 (tested on linux/win32) > > > > > > > > - split out pytest independent process support into xprocess.py > > > > > > > > - add info.kill() method and some smaller refactoring > > > > > > > > - fix various windows related Popen / killing details > > > > > > > > - add tests > > > > > > > > > > > > pytest-pep8: > > > > > > > > - use pytest-2.4.2 node.add_marker() API for adding "pep8" marker > > > > > > > > _______________________________________________ > > > > Pytest-dev mailing list > > > > Pytest-dev at python.org > > > > https://mail.python.org/mailman/listinfo/pytest-dev > > > > > > From nicoddemus at gmail.com Fri Oct 4 23:43:33 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Fri, 4 Oct 2013 18:43:33 -0300 Subject: [pytest-dev] (new) pytest plugins / note to plugin authors In-Reply-To: <20131004212520.GJ14010@merlinux.eu> References: <20131004130933.GE14010@merlinux.eu> <20131004194531.GH14010@merlinux.eu> <20131004212520.GJ14010@merlinux.eu> Message-ID: Hi Holger, On Fri, Oct 4, 2013 at 6:25 PM, holger krekel wrote: > > Are you interested in displaying compatibility to several py.test > versions, > > or only to the latest one? > > ideally latest release and trunk ... > Fair enough. :) > > and some way to re-generate this information (especially the test > status). > > > I guess we could start with considering plugins that have a tox.ini > > > and then write a script to download the plugin, unpack it, run tests > etc. > > > > > > > I considered this, but then I realize we might have to implement several > > features to support that, like installing dependencies: pytest-qt for > > instance requires PySide, which must be installed for it to run its own > > tests. Other plugins might have other requirements to run its tests (like > > having a browser open for headless testing), which would have to be > > supported as well and so on. I fear we would have to implement a lot of > > infrastructure similar to what travis-ci.org or d > > rone.iodoes for example. > > > > Perhaps we could adopt a solution similar to coveralls.io, in which the > > author is responsible to upload coverage data, and the site collects and > > displays that information. We could provide a script or code snippet to > be > > included by the plugin author in he's C.I of choice that would generate > and > > upload pytest-compatibility information. This could be easily hosted in a > > PaaS like Heroku. > > > > We would have to count on plugin authors' contribution to the system for > it > > to work on this scheme, of course. > > I wonder how difficult most plugins really are to setup. > PyQT might be difficult but OTOH many others should just be fine. > > And pushing the responsibility for performing tests and uploading results > still would not get the benefit of testing current plugin releases > against a new core pytest release (which can break many plugins whereas > a new plugin release can just break that plugin :). Yes, having to rely on package authors to update their CI system to use a new pytest version would defeat the purpose of the status page. :/ > So i'd think it's best to check how many of the plugins even provide tests. > Hmm, of that I have no idea... But even just collecting the other information i mentioned (without > the test status / running questions) would already be helpful, btw. > That seems simple enough, a script that queries pypi and extracts this information could be used to generate a page and/or statistics. >From my understanding you would like a page to accomplish two things: 1. A list of known py-test plugins and their py27/py33 status. 2. If the plugins work against latest pytest and against trunk, so pytest developers know if they're breaking existing plugins (intentionally or not). Is that correct? I'm trying to figure out a semi automatic way to accomplish both. :) Cheers, Bruno > cheers, > holger > > > The other part is how to present this info, but e.g. a CSV file > > > can be rendered by sphinx and the main docs could include it > > > or we make a separate page if need be. > > > > > > > thanks for considering to help! > > > > > > > Glad to. :) > > > > Best Regards, > > Bruno > > > > > > > > holger > > > > > > > Cheers, > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 4, 2013 at 10:09 AM, holger krekel > > > wrote: > > > > > > > > > > > > > > I just did three pytest plugin releases to get out a few minor > > > > > fixes and improvements. see below. As far as i am aware most > > > > > existing plugins (as well as the new releases here) should be > > > > > compatible to pytest-2.4.2. Some plugins like pytest-capturelog > > > > > have seen their latest release three years ago but remain > functional. > > > > > A good sign that the plugin system is reasonable i guess. > > > > > > > > > > If someone wants to help with organizing plugin compatibility > > > > > tests and maintaining a status page, i'd really appreciate that. > > > > > With over 50 existing plugins it's not easy to stay on top. > > > > > The current pytest.org page is only a start: > > > > > > > > > > http://pytest.org/latest/plugins.html#extplugins > > > > > > > > > > and here is the full pypi list: > > > > > > > > > > > > > > > > > > > https://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search > > > > > > > > > > there are also some github/bitbucket-only plugins, in addition. > > > > > > > > > > And finally a note to plugin authors: please be subscribed > > > > > to pytest-dev and post new plugin releases here. > > > > > > > > > > best, > > > > > holger > > > > > > > > > > pytest-xdist-1.9: > > > > > > > > > > - changed LICENSE to MIT > > > > > > > > > > - fix duplicate reported test ids with --looponfailing > > > > > (thanks Jeremy Thurgood) > > > > > > > > > > - fix pytest issue41: re-run tests on all file changes, not just > > > > > randomly select ones like .py/.c. > > > > > > > > > > - fix pytest issue347: slaves running on top of Python3.2 > > > > > will set PYTHONDONTWRITEYBTECODE to 1 to avoid import concurrency > > > > > bugs. > > > > > > > > > > pytest-xprocess-0.8: > > > > > > > > > > - support python3 (tested on linux/win32) > > > > > > > > > > - split out pytest independent process support into xprocess.py > > > > > > > > > > - add info.kill() method and some smaller refactoring > > > > > > > > > > - fix various windows related Popen / killing details > > > > > > > > > > - add tests > > > > > > > > > > > > > > > pytest-pep8: > > > > > > > > > > - use pytest-2.4.2 node.add_marker() API for adding "pep8" marker > > > > > > > > > > _______________________________________________ > > > > > Pytest-dev mailing list > > > > > Pytest-dev at python.org > > > > > https://mail.python.org/mailman/listinfo/pytest-dev > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Sat Oct 5 00:18:14 2013 From: holger at merlinux.eu (holger krekel) Date: Fri, 4 Oct 2013 22:18:14 +0000 Subject: [pytest-dev] (new) pytest plugins / note to plugin authors In-Reply-To: References: <20131004130933.GE14010@merlinux.eu> <20131004194531.GH14010@merlinux.eu> <20131004212520.GJ14010@merlinux.eu> Message-ID: <20131004221814.GK14010@merlinux.eu> On Fri, Oct 04, 2013 at 18:43 -0300, Bruno Oliveira wrote: > But even just collecting the other information i mentioned (without > > the test status / running questions) would already be helpful, btw. > > > > That seems simple enough, a script that queries pypi and extracts this > information could be used to generate a page and/or statistics. > > >From my understanding you would like a page to accomplish two things: > > 1. A list of known py-test plugins and their py27/py33 status. > 2. If the plugins work against latest pytest and against trunk, so pytest > developers know if they're breaking existing plugins (intentionally or not). > > Is that correct? yes. Along with repo information (probably not easy to automatically detect but doesn't change often so once manually is fine). > I'm trying to figure out a semi automatic way to accomplish both. :) You can use pypixmlrpc, maybe, to get at meta info (but not something like repository). cheers, holger > Cheers, > Bruno > > > > cheers, > > holger > > > > > The other part is how to present this info, but e.g. a CSV file > > > > can be rendered by sphinx and the main docs could include it > > > > or we make a separate page if need be. > > > > > > > > > > thanks for considering to help! > > > > > > > > > > Glad to. :) > > > > > > Best Regards, > > > Bruno > > > > > > > > > > > holger > > > > > > > > > Cheers, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 4, 2013 at 10:09 AM, holger krekel > > > > wrote: > > > > > > > > > > > > > > > > > I just did three pytest plugin releases to get out a few minor > > > > > > fixes and improvements. see below. As far as i am aware most > > > > > > existing plugins (as well as the new releases here) should be > > > > > > compatible to pytest-2.4.2. Some plugins like pytest-capturelog > > > > > > have seen their latest release three years ago but remain > > functional. > > > > > > A good sign that the plugin system is reasonable i guess. > > > > > > > > > > > > If someone wants to help with organizing plugin compatibility > > > > > > tests and maintaining a status page, i'd really appreciate that. > > > > > > With over 50 existing plugins it's not easy to stay on top. > > > > > > The current pytest.org page is only a start: > > > > > > > > > > > > http://pytest.org/latest/plugins.html#extplugins > > > > > > > > > > > > and here is the full pypi list: > > > > > > > > > > > > > > > > > > > > > > > > https://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search > > > > > > > > > > > > there are also some github/bitbucket-only plugins, in addition. > > > > > > > > > > > > And finally a note to plugin authors: please be subscribed > > > > > > to pytest-dev and post new plugin releases here. > > > > > > > > > > > > best, > > > > > > holger > > > > > > > > > > > > pytest-xdist-1.9: > > > > > > > > > > > > - changed LICENSE to MIT > > > > > > > > > > > > - fix duplicate reported test ids with --looponfailing > > > > > > (thanks Jeremy Thurgood) > > > > > > > > > > > > - fix pytest issue41: re-run tests on all file changes, not just > > > > > > randomly select ones like .py/.c. > > > > > > > > > > > > - fix pytest issue347: slaves running on top of Python3.2 > > > > > > will set PYTHONDONTWRITEYBTECODE to 1 to avoid import concurrency > > > > > > bugs. > > > > > > > > > > > > pytest-xprocess-0.8: > > > > > > > > > > > > - support python3 (tested on linux/win32) > > > > > > > > > > > > - split out pytest independent process support into xprocess.py > > > > > > > > > > > > - add info.kill() method and some smaller refactoring > > > > > > > > > > > > - fix various windows related Popen / killing details > > > > > > > > > > > > - add tests > > > > > > > > > > > > > > > > > > pytest-pep8: > > > > > > > > > > > > - use pytest-2.4.2 node.add_marker() API for adding "pep8" marker > > > > > > > > > > > > _______________________________________________ > > > > > > Pytest-dev mailing list > > > > > > Pytest-dev at python.org > > > > > > https://mail.python.org/mailman/listinfo/pytest-dev > > > > > > > > > > > > From nicoddemus at gmail.com Thu Oct 10 02:44:41 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Wed, 9 Oct 2013 21:44:41 -0300 Subject: [pytest-dev] (new) pytest plugins / note to plugin authors In-Reply-To: <20131004221814.GK14010@merlinux.eu> References: <20131004130933.GE14010@merlinux.eu> <20131004194531.GH14010@merlinux.eu> <20131004212520.GJ14010@merlinux.eu> <20131004221814.GK14010@merlinux.eu> Message-ID: Hi Holger, I've played around a bit with PyPi's api. Obtaining the meta data is easy enough, but I found that only 7 out of 60 of the packages (searching for "pytest-") have a tox.ini file. Even thought it is just a few, I think we can use that initially especially if that encourages more plugin authors to use tox to manage its test runs. I'm thinking of putting a script that can generate Sphinx docs from live PyPi package information, and later on be also able to test plugins compatibility. I'm thinking to provide a patch to pytest's own documentation, but later on we can move this to a live app that periodically fetches that information to keep it up to date if you think it is necessary. How does that sound? Cheers, Bruno On Fri, Oct 4, 2013 at 7:18 PM, holger krekel wrote: > On Fri, Oct 04, 2013 at 18:43 -0300, Bruno Oliveira wrote: > > But even just collecting the other information i mentioned (without > > > the test status / running questions) would already be helpful, btw. > > > > > > > That seems simple enough, a script that queries pypi and extracts this > > information could be used to generate a page and/or statistics. > > > > >From my understanding you would like a page to accomplish two things: > > > > 1. A list of known py-test plugins and their py27/py33 status. > > 2. If the plugins work against latest pytest and against trunk, so pytest > > developers know if they're breaking existing plugins (intentionally or > not). > > > > Is that correct? > > yes. Along with repo information (probably not easy to automatically > detect but doesn't change often so once manually is fine). > > > I'm trying to figure out a semi automatic way to accomplish both. :) > > You can use pypixmlrpc, maybe, to get at meta info (but not something > like repository). > > cheers, > holger > > > Cheers, > > Bruno > > > > > > > cheers, > > > holger > > > > > > > The other part is how to present this info, but e.g. a CSV file > > > > > can be rendered by sphinx and the main docs could include it > > > > > or we make a separate page if need be. > > > > > > > > > > > > > thanks for considering to help! > > > > > > > > > > > > > Glad to. :) > > > > > > > > Best Regards, > > > > Bruno > > > > > > > > > > > > > > holger > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 4, 2013 at 10:09 AM, holger krekel < > holger at merlinux.eu> > > > > > wrote: > > > > > > > > > > > > > > > > > > > > I just did three pytest plugin releases to get out a few minor > > > > > > > fixes and improvements. see below. As far as i am aware most > > > > > > > existing plugins (as well as the new releases here) should be > > > > > > > compatible to pytest-2.4.2. Some plugins like > pytest-capturelog > > > > > > > have seen their latest release three years ago but remain > > > functional. > > > > > > > A good sign that the plugin system is reasonable i guess. > > > > > > > > > > > > > > If someone wants to help with organizing plugin compatibility > > > > > > > tests and maintaining a status page, i'd really appreciate > that. > > > > > > > With over 50 existing plugins it's not easy to stay on top. > > > > > > > The current pytest.org page is only a start: > > > > > > > > > > > > > > http://pytest.org/latest/plugins.html#extplugins > > > > > > > > > > > > > > and here is the full pypi list: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search > > > > > > > > > > > > > > there are also some github/bitbucket-only plugins, in addition. > > > > > > > > > > > > > > And finally a note to plugin authors: please be subscribed > > > > > > > to pytest-dev and post new plugin releases here. > > > > > > > > > > > > > > best, > > > > > > > holger > > > > > > > > > > > > > > pytest-xdist-1.9: > > > > > > > > > > > > > > - changed LICENSE to MIT > > > > > > > > > > > > > > - fix duplicate reported test ids with --looponfailing > > > > > > > (thanks Jeremy Thurgood) > > > > > > > > > > > > > > - fix pytest issue41: re-run tests on all file changes, not > just > > > > > > > randomly select ones like .py/.c. > > > > > > > > > > > > > > - fix pytest issue347: slaves running on top of Python3.2 > > > > > > > will set PYTHONDONTWRITEYBTECODE to 1 to avoid import > concurrency > > > > > > > bugs. > > > > > > > > > > > > > > pytest-xprocess-0.8: > > > > > > > > > > > > > > - support python3 (tested on linux/win32) > > > > > > > > > > > > > > - split out pytest independent process support into xprocess.py > > > > > > > > > > > > > > - add info.kill() method and some smaller refactoring > > > > > > > > > > > > > > - fix various windows related Popen / killing details > > > > > > > > > > > > > > - add tests > > > > > > > > > > > > > > > > > > > > > pytest-pep8: > > > > > > > > > > > > > > - use pytest-2.4.2 node.add_marker() API for adding "pep8" > marker > > > > > > > > > > > > > > _______________________________________________ > > > > > > > Pytest-dev mailing list > > > > > > > Pytest-dev at python.org > > > > > > > https://mail.python.org/mailman/listinfo/pytest-dev > > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Thu Oct 10 09:22:27 2013 From: holger at merlinux.eu (holger krekel) Date: Thu, 10 Oct 2013 07:22:27 +0000 Subject: [pytest-dev] (new) pytest plugins / note to plugin authors In-Reply-To: References: <20131004130933.GE14010@merlinux.eu> <20131004194531.GH14010@merlinux.eu> <20131004212520.GJ14010@merlinux.eu> <20131004221814.GK14010@merlinux.eu> Message-ID: <20131010072227.GB14010@merlinux.eu> Hi Bruno, On Wed, Oct 09, 2013 at 21:44 -0300, Bruno Oliveira wrote: > Hi Holger, > > I've played around a bit with PyPi's api. > > Obtaining the meta data is easy enough, but I found that only 7 out of 60 > of the packages (searching for "pytest-") have a tox.ini file. Even thought > it is just a few, I think we can use that initially especially if that > encourages more plugin authors to use tox to manage its test runs. We could think about generating a tox.ini which runs "py.test -h" afterwards and thus verifies that the plugin installs, at least. > I'm thinking of putting a script that can generate Sphinx docs from live > PyPi package information, and later on be also able to test plugins > compatibility. I'm thinking to provide a patch to pytest's own > documentation, but later on we can move this to a live app that > periodically fetches that information to keep it up to date if you think it > is necessary. How does that sound? A script that generates sphinx-doc(s) makes sense. Maybe to finally produce content at http://pytest.org/latest/plugins/ or so and we make a direct navigation link to its index page. If you have an entry point into that script that allows to re-check a particular project we could think about calling that triggered by the PyPI changelog API (i can do that part as i have been working with this stuff a lot in the devpi.net project), or other events like preparing a new pytest release. As this script will -- when installing or testing -- execute almost arbitrary code released to pypi i think we should have a human-screened "whitelist" of projects we inspect/try in this manner. I'll give you commit rights to the pytest repo once the initial script produces something. cheers and thanks for the thoughts and efforts! holger From holger at merlinux.eu Fri Oct 11 10:40:45 2013 From: holger at merlinux.eu (holger krekel) Date: Fri, 11 Oct 2013 08:40:45 +0000 Subject: [pytest-dev] solving the "too static" fixture scope problem Message-ID: <20131011084045.GD14010@merlinux.eu> Hi pytest users and developers, I'd like to discuss and get feedback on extending the fixture mechanism and fix what i consider a biggest limitation at the moment. Problem: fixtures are tied statically to a scope ================================================= For example, you cannot use monkeypatch in a higher than "function" scoped fixture. Same is true for tmpdir and probably also many user-defined fixtures. I've certainly had this problem myself many times that i had a fixture function that didn't really care in what scope it was used. There are ways to get around this problem but they are not pretty: @pytest.fixture(scope="module") def myfix_module(request return _myfix(request) @pytest.fixture(scope="function") def myfix_function(request return _myfix(request) where _myfix is the function that doesn't care about the actual scope. Even then, you can't use builtin fixtures like "monkeypatch", "tmpdir", etc. Solution Idea: introduce "each" scoped fixtures ===================================================== The idea is allow a fixture function to declare it wants to be used in the scope of the requesting fixture function (or at function-scope if used from a test). This is how "monkeypatch" would be implemented then: @pytest.fixture(scope="each") def monkeypatch(request): ... # same implementation as in _pytest/monkeypatch.py The new "each" scope means that each fixture/test requesting the "monkeypatch" fixture would receive its own fixture instance. So a session-scoped fixture could naturally use it like this: @pytest.fixture(scope="session") def myfix(monkeypatch): monkeypatch.setattr(...) return some_value The passed in monkeypatch object here is a specific instance just for the ``myfix`` fixture function: "each" fixture function requesting ``monkeypatch`` gets a new instance of it. If e.g. a test uses another module-scoped fixture defined like this: @pytest.fixture(scope="module") def myfix2(monkeypatch): mp.setattr(...) return some_value this would invoke the ``monkeypatch`` fixture function a second time, resulting in a new instance for use by the ``myfix2`` instance. The same logic could be applied to other fixtures like "tmpdir" or user-defined ones. Do you like this idea? Would you find it helpful for your test suites? There is one issue i am not sure about yet, however. Currently, when a test requires fixture A and B, and B requires C and C requires A, then the two "A" would be exactly the same object, independently of what which scopes are declared. If A=="tmpdir", then the test's tmpdir and C's tmpdir would be the same directory. I often don't find this desirable. If tmpdir would be an "each" scoped fixture, then C and the test would each receive a clean new tmpdir. If that is a backward-compat issue, we could introduce another name for the new "each" scoped tmpdir. I usually find myself working around the problem of a "tmpdir" shared by multiple different fixtures, though. cheers, holger From florian.schulze at gmx.net Fri Oct 11 12:51:32 2013 From: florian.schulze at gmx.net (Florian Schulze) Date: Fri, 11 Oct 2013 12:51:32 +0200 Subject: [pytest-dev] solving the "too static" fixture scope problem In-Reply-To: <20131011084045.GD14010@merlinux.eu> References: <20131011084045.GD14010@merlinux.eu> Message-ID: <3FD2A0D1-78B7-46AE-8B52-63958919032B@gmx.net> I would expect the behaviour you describe for scope="each" to be the default for a fixture without a scope. Why introduce a new scope for that? Am I missing something? I haven't used pytest for such complex things yet. Regarding the backward compatibility issue you described, I would expect the new behaviour instead of the current one. But maybe I'm missing something, because of the abstract example. Regards, Florian Schulze On 11.10.2013, at 10:40, holger krekel wrote: > Hi pytest users and developers, > > I'd like to discuss and get feedback on > extending the fixture mechanism and fix what > i consider a biggest limitation at the moment. > > Problem: fixtures are tied statically to a scope > ================================================= > > For example, you cannot use monkeypatch in a higher > than "function" scoped fixture. Same is true for > tmpdir and probably also many user-defined fixtures. > I've certainly had this problem myself many times > that i had a fixture function that didn't really > care in what scope it was used. There are > ways to get around this problem but they are not > pretty: > > @pytest.fixture(scope="module") > def myfix_module(request > return _myfix(request) > > @pytest.fixture(scope="function") > def myfix_function(request > return _myfix(request) > > where _myfix is the function that doesn't > care about the actual scope. Even then, you > can't use builtin fixtures like "monkeypatch", > "tmpdir", etc. > > Solution Idea: introduce "each" scoped fixtures > ===================================================== > > The idea is allow a fixture function to declare it wants > to be used in the scope of the requesting fixture function > (or at function-scope if used from a test). > > This is how "monkeypatch" would be implemented then: > > @pytest.fixture(scope="each") > def monkeypatch(request): > ... # same implementation as in _pytest/monkeypatch.py > > The new "each" scope means that each fixture/test requesting > the "monkeypatch" fixture would receive its own fixture instance. > > So a session-scoped fixture could naturally use it like this: > > @pytest.fixture(scope="session") > def myfix(monkeypatch): > monkeypatch.setattr(...) > return some_value > > The passed in monkeypatch object here is a specific instance just > for the ``myfix`` fixture function: "each" fixture function > requesting ``monkeypatch`` gets a new instance of it. > If e.g. a test uses another module-scoped fixture defined like this: > > @pytest.fixture(scope="module") > def myfix2(monkeypatch): > mp.setattr(...) > return some_value > > this would invoke the ``monkeypatch`` fixture function a second time, > resulting in a new instance for use by the ``myfix2`` instance. > > The same logic could be applied to other fixtures > like "tmpdir" or user-defined ones. > > Do you like this idea? Would you find it helpful for your test suites? > > There is one issue i am not sure about yet, however. Currently, > when a test requires fixture A and B, and B requires C and C requires A, > then the two "A" would be exactly the same object, independently of what > which scopes are declared. If A=="tmpdir", then the test's tmpdir and > C's tmpdir would be the same directory. I often don't find this desirable. > If tmpdir would be an "each" scoped fixture, then C and the test would > each receive a clean new tmpdir. If that is a backward-compat issue, > we could introduce another name for the new "each" scoped tmpdir. > I usually find myself working around the problem of a "tmpdir" > shared by multiple different fixtures, though. > > cheers, > holger > _______________________________________________ > Pytest-dev mailing list > Pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev From holger at merlinux.eu Fri Oct 11 12:57:24 2013 From: holger at merlinux.eu (holger krekel) Date: Fri, 11 Oct 2013 10:57:24 +0000 Subject: [pytest-dev] solving the "too static" fixture scope problem In-Reply-To: <3FD2A0D1-78B7-46AE-8B52-63958919032B@gmx.net> References: <20131011084045.GD14010@merlinux.eu> <3FD2A0D1-78B7-46AE-8B52-63958919032B@gmx.net> Message-ID: <20131011105724.GE14010@merlinux.eu> On Fri, Oct 11, 2013 at 12:51 +0200, Florian Schulze wrote: > I would expect the behaviour you describe for scope="each" to be the default for a fixture without a scope. Why introduce a new scope for that? Am I missing something? I haven't used pytest for such complex things yet. Defaulting to scope="each" is an interesting proposition but it would be largely backward-compatible and i think in real situations. Consider a fixture like this: @pytest.fixture def db(...): return DbInstance() and then a test and other fixtures using this "db" fixture. They would all get distinct instances (people using fixtures more extensively implicitly rely on the current per-test "sharing" of resources and i think it makes sense, just not for all resources). > Regarding the backward compatibility issue you described, I would expect the new behaviour instead of the current one. But maybe I'm missing something, because of the abstract example. If you refer to tmpdir/monkeypatch and potentially others, i agree that the current behaviour is more surprising (sharing the tmpdir across multiple fixtures which don't even neccessarily know about each other). cheers, holger > Regards, > Florian Schulze > > On 11.10.2013, at 10:40, holger krekel wrote: > > > Hi pytest users and developers, > > > > I'd like to discuss and get feedback on > > extending the fixture mechanism and fix what > > i consider a biggest limitation at the moment. > > > > Problem: fixtures are tied statically to a scope > > ================================================= > > > > For example, you cannot use monkeypatch in a higher > > than "function" scoped fixture. Same is true for > > tmpdir and probably also many user-defined fixtures. > > I've certainly had this problem myself many times > > that i had a fixture function that didn't really > > care in what scope it was used. There are > > ways to get around this problem but they are not > > pretty: > > > > @pytest.fixture(scope="module") > > def myfix_module(request > > return _myfix(request) > > > > @pytest.fixture(scope="function") > > def myfix_function(request > > return _myfix(request) > > > > where _myfix is the function that doesn't > > care about the actual scope. Even then, you > > can't use builtin fixtures like "monkeypatch", > > "tmpdir", etc. > > > > Solution Idea: introduce "each" scoped fixtures > > ===================================================== > > > > The idea is allow a fixture function to declare it wants > > to be used in the scope of the requesting fixture function > > (or at function-scope if used from a test). > > > > This is how "monkeypatch" would be implemented then: > > > > @pytest.fixture(scope="each") > > def monkeypatch(request): > > ... # same implementation as in _pytest/monkeypatch.py > > > > The new "each" scope means that each fixture/test requesting > > the "monkeypatch" fixture would receive its own fixture instance. > > > > So a session-scoped fixture could naturally use it like this: > > > > @pytest.fixture(scope="session") > > def myfix(monkeypatch): > > monkeypatch.setattr(...) > > return some_value > > > > The passed in monkeypatch object here is a specific instance just > > for the ``myfix`` fixture function: "each" fixture function > > requesting ``monkeypatch`` gets a new instance of it. > > If e.g. a test uses another module-scoped fixture defined like this: > > > > @pytest.fixture(scope="module") > > def myfix2(monkeypatch): > > mp.setattr(...) > > return some_value > > > > this would invoke the ``monkeypatch`` fixture function a second time, > > resulting in a new instance for use by the ``myfix2`` instance. > > > > The same logic could be applied to other fixtures > > like "tmpdir" or user-defined ones. > > > > Do you like this idea? Would you find it helpful for your test suites? > > > > There is one issue i am not sure about yet, however. Currently, > > when a test requires fixture A and B, and B requires C and C requires A, > > then the two "A" would be exactly the same object, independently of what > > which scopes are declared. If A=="tmpdir", then the test's tmpdir and > > C's tmpdir would be the same directory. I often don't find this desirable. > > If tmpdir would be an "each" scoped fixture, then C and the test would > > each receive a clean new tmpdir. If that is a backward-compat issue, > > we could introduce another name for the new "each" scoped tmpdir. > > I usually find myself working around the problem of a "tmpdir" > > shared by multiple different fixtures, though. > > > > cheers, > > holger > > _______________________________________________ > > Pytest-dev mailing list > > Pytest-dev at python.org > > https://mail.python.org/mailman/listinfo/pytest-dev > From nicoddemus at gmail.com Fri Oct 11 13:05:03 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Fri, 11 Oct 2013 08:05:03 -0300 Subject: [pytest-dev] (new) pytest plugins / note to plugin authors In-Reply-To: <20131010072227.GB14010@merlinux.eu> References: <20131004130933.GE14010@merlinux.eu> <20131004194531.GH14010@merlinux.eu> <20131004212520.GJ14010@merlinux.eu> <20131004221814.GK14010@merlinux.eu> <20131010072227.GB14010@merlinux.eu> Message-ID: Hi Holger, On Thu, Oct 10, 2013 at 4:22 AM, holger krekel wrote: > > On Wed, Oct 09, 2013 at 21:44 -0300, Bruno Oliveira wrote: > > > Obtaining the meta data is easy enough, but I found that only 7 out of 60 > > of the packages (searching for "pytest-") have a tox.ini file. Even > thought > > it is just a few, I think we can use that initially especially if that > > encourages more plugin authors to use tox to manage its test runs. > > We could think about generating a tox.ini which runs "py.test -h" > afterwards > and thus verifies that the plugin installs, at least. > Good, that would work at least as a smoke test. :) > > I'm thinking of putting a script that can generate Sphinx docs from live > > PyPi package information, and later on be also able to test plugins > > compatibility. I'm thinking to provide a patch to pytest's own > > documentation, but later on we can move this to a live app that > > periodically fetches that information to keep it up to date if you think > it > > is necessary. How does that sound? > > A script that generates sphinx-doc(s) makes sense. Maybe to finally > produce content at http://pytest.org/latest/plugins/ or so and we make > a direct navigation link to its index page. > I'm working on it and will create a PR so we can discuss over the code. If you have an entry point into that script that allows to re-check a > particular project we could think about calling that triggered by the PyPI > changelog API (i can do that part as i have been working with this stuff > a lot in the devpi.net project), or other events like preparing a new > pytest release. > > As this script will -- when installing or testing -- execute almost > arbitrary code released to pypi i think we should have a human-screened > "whitelist" of projects we inspect/try in this manner. > Yes, I have been thinking about this and will make some experiments in that regard. (I didn't know about devpi.net, will check it out) Cheers, Bruno. -------------- next part -------------- An HTML attachment was scrubbed... URL: From florian.schulze at gmx.net Fri Oct 11 13:09:26 2013 From: florian.schulze at gmx.net (Florian Schulze) Date: Fri, 11 Oct 2013 13:09:26 +0200 Subject: [pytest-dev] solving the "too static" fixture scope problem In-Reply-To: <20131011105724.GE14010@merlinux.eu> References: <20131011084045.GD14010@merlinux.eu> <3FD2A0D1-78B7-46AE-8B52-63958919032B@gmx.net> <20131011105724.GE14010@merlinux.eu> Message-ID: On 11.10.2013, at 12:57, holger krekel wrote: > On Fri, Oct 11, 2013 at 12:51 +0200, Florian Schulze wrote: >> I would expect the behaviour you describe for scope="each" to be the default for a fixture without a scope. Why introduce a new scope for that? Am I missing something? I haven't used pytest for such complex things yet. > > Defaulting to scope="each" is an interesting proposition but it > would be largely backward-compatible and i think in real situations. > Consider a fixture like this: > > @pytest.fixture > def db(...): > return DbInstance() > > and then a test and other fixtures using this "db" fixture. They > would all get distinct instances (people using fixtures more extensively > implicitly rely on the current per-test "sharing" of resources and i think > it makes sense, just not for all resources). Ah, I think I get the difference now. Regards, Florian Schulze From vladimir at keleshev.com Fri Oct 11 13:14:00 2013 From: vladimir at keleshev.com (Vladimir Keleshev) Date: Fri, 11 Oct 2013 13:14:00 +0200 Subject: [pytest-dev] solving the "too static" fixture scope problem In-Reply-To: <20131011105724.GE14010@merlinux.eu> References: <20131011084045.GD14010@merlinux.eu> <3FD2A0D1-78B7-46AE-8B52-63958919032B@gmx.net> <20131011105724.GE14010@merlinux.eu> Message-ID: <293531381490040@web30m.yandex.ru> > If you refer to tmpdir/monkeypatch and potentially others, i agree > that the current behaviour is more surprising (sharing the tmpdir > across multiple fixtures which don't even neccessarily know about each > other). Absolutely. Was a major WAT for me. ?Vladimir 11.10.2013, 12:57, "holger krekel" : > On Fri, Oct 11, 2013 at 12:51 +0200, Florian Schulze wrote: > >> ?I would expect the behaviour you describe for scope="each" to be the default for a fixture without a scope. Why introduce a new scope for that? Am I missing something? I haven't used pytest for such complex things yet. > > Defaulting to scope="each" is an interesting proposition but it > would be largely backward-compatible and i think in real situations. > Consider a fixture like this: > > ????@pytest.fixture > ????def db(...): > ????????return DbInstance() > > and then a test and other fixtures using this "db" fixture. ?They > would all get distinct instances (people using fixtures more extensively > implicitly rely on the current per-test "sharing" of resources and i think > it makes sense, just not for all resources). > >> ?Regarding the backward compatibility issue you described, I would expect the new behaviour instead of the current one. But maybe I'm missing something, because of the abstract example. > > If you refer to tmpdir/monkeypatch and potentially others, i agree > that the current behaviour is more surprising (sharing the tmpdir > across multiple fixtures which don't even neccessarily know about each > other). > > cheers, > holger > >> ?Regards, >> ?Florian Schulze >> >> ?On 11.10.2013, at 10:40, holger krekel wrote: >>> ?Hi pytest users and developers, >>> >>> ?I'd like to discuss and get feedback on >>> ?extending the fixture mechanism and fix what >>> ?i consider a biggest limitation at the moment. >>> >>> ?Problem: fixtures are tied statically to a scope >>> ?================================================= >>> >>> ?For example, you cannot use monkeypatch in a higher >>> ?than "function" scoped fixture. Same is true for >>> ?tmpdir and probably also many user-defined fixtures. >>> ?I've certainly had this problem myself many times >>> ?that i had a fixture function that didn't really >>> ?care in what scope it was used. ?There are >>> ?ways to get around this problem but they are not >>> ?pretty: >>> >>> ????@pytest.fixture(scope="module") >>> ????def myfix_module(request >>> ????????return _myfix(request) >>> >>> ????@pytest.fixture(scope="function") >>> ????def myfix_function(request >>> ????????return _myfix(request) >>> >>> ?where _myfix is the function that doesn't >>> ?care about the actual scope. ?Even then, you >>> ?can't use builtin fixtures like "monkeypatch", >>> ?"tmpdir", etc. >>> >>> ?Solution Idea: introduce "each" scoped fixtures >>> ?===================================================== >>> >>> ?The idea is allow a fixture function to declare it wants >>> ?to be used in the scope of the requesting fixture function >>> ?(or at function-scope if used from a test). >>> >>> ?This is how "monkeypatch" would be implemented then: >>> >>> ????@pytest.fixture(scope="each") >>> ????def monkeypatch(request): >>> ????????... ?# same implementation as in _pytest/monkeypatch.py >>> >>> ?The new "each" scope means that each fixture/test requesting >>> ?the "monkeypatch" fixture would receive its own fixture instance. >>> >>> ?So a session-scoped fixture could naturally use it like this: >>> >>> ????@pytest.fixture(scope="session") >>> ????def myfix(monkeypatch): >>> ????????monkeypatch.setattr(...) >>> ????????return some_value >>> >>> ?The passed in monkeypatch object here is a specific instance just >>> ?for the ``myfix`` fixture function: "each" fixture function >>> ?requesting ``monkeypatch`` gets a new instance of it. >>> ?If e.g. a test uses another module-scoped fixture defined like this: >>> >>> ????@pytest.fixture(scope="module") >>> ????def myfix2(monkeypatch): >>> ????????mp.setattr(...) >>> ????????return some_value >>> >>> ?this would invoke the ``monkeypatch`` fixture function a second time, >>> ?resulting in a new instance for use by the ``myfix2`` instance. >>> >>> ?The same logic could be applied to other fixtures >>> ?like "tmpdir" or user-defined ones. >>> >>> ?Do you like this idea? Would you find it helpful for your test suites? >>> >>> ?There is one issue i am not sure about yet, however. ?Currently, >>> ?when a test requires fixture A and B, and B requires C and C requires A, >>> ?then the two "A" would be exactly the same object, independently of what >>> ?which scopes are declared. ?If A=="tmpdir", then the test's tmpdir and >>> ?C's tmpdir would be the same directory. ?I often don't find this desirable. >>> ?If tmpdir would be an "each" scoped fixture, then C and the test would >>> ?each receive a clean new tmpdir. ?If that is a backward-compat issue, >>> ?we could introduce another name for the new "each" scoped tmpdir. >>> ?I usually find myself working around the problem of a "tmpdir" >>> ?shared by multiple different fixtures, though. >>> >>> ?cheers, >>> ?holger >>> ?_______________________________________________ >>> ?Pytest-dev mailing list >>> ?Pytest-dev at python.org >>> ?https://mail.python.org/mailman/listinfo/pytest-dev > > _______________________________________________ > Pytest-dev mailing list > Pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev From nicoddemus at gmail.com Fri Oct 11 13:25:32 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Fri, 11 Oct 2013 08:25:32 -0300 Subject: [pytest-dev] solving the "too static" fixture scope problem In-Reply-To: <293531381490040@web30m.yandex.ru> References: <20131011084045.GD14010@merlinux.eu> <3FD2A0D1-78B7-46AE-8B52-63958919032B@gmx.net> <20131011105724.GE14010@merlinux.eu> <293531381490040@web30m.yandex.ru> Message-ID: On Fri, Oct 11, 2013 at 8:14 AM, Vladimir Keleshev wrote: > > If you refer to tmpdir/monkeypatch and potentially others, i agree > > that the current behaviour is more surprising (sharing the tmpdir > > across multiple fixtures which don't even neccessarily know about each > > other). > > Absolutely. Was a major WAT for me. > Weird, to me it was the exact opposite about tmpdir: it is a common usage in our code base to create a temporary directory for usage through a test suite (usually a module containing related tests). And different fixtures sharing tmpdir makes sense in this scenario. Cheers, Bruno. -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Fri Oct 11 13:35:24 2013 From: holger at merlinux.eu (holger krekel) Date: Fri, 11 Oct 2013 11:35:24 +0000 Subject: [pytest-dev] solving the "too static" fixture scope problem In-Reply-To: References: <20131011084045.GD14010@merlinux.eu> <3FD2A0D1-78B7-46AE-8B52-63958919032B@gmx.net> <20131011105724.GE14010@merlinux.eu> <293531381490040@web30m.yandex.ru> Message-ID: <20131011113524.GF14010@merlinux.eu> On Fri, Oct 11, 2013 at 08:25 -0300, Bruno Oliveira wrote: > On Fri, Oct 11, 2013 at 8:14 AM, Vladimir Keleshev wrote: > > > > If you refer to tmpdir/monkeypatch and potentially others, i agree > > > that the current behaviour is more surprising (sharing the tmpdir > > > across multiple fixtures which don't even neccessarily know about each > > > other). > > > > Absolutely. Was a major WAT for me. > > > > Weird, to me it was the exact opposite about tmpdir: it is a common usage > in our code base to create a temporary directory for usage through a test > suite (usually a module containing related tests). And different fixtures > sharing tmpdir makes sense in this scenario. Are you really talking about the pytest fixture "tmpdir" here? "Throughout the test suite" sounds like something else because "tmpdir" is function scoped and created afresh for each test function requesting it. holger From nicoddemus at gmail.com Fri Oct 11 15:51:30 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Fri, 11 Oct 2013 10:51:30 -0300 Subject: [pytest-dev] solving the "too static" fixture scope problem In-Reply-To: <20131011113524.GF14010@merlinux.eu> References: <20131011084045.GD14010@merlinux.eu> <3FD2A0D1-78B7-46AE-8B52-63958919032B@gmx.net> <20131011105724.GE14010@merlinux.eu> <293531381490040@web30m.yandex.ru> <20131011113524.GF14010@merlinux.eu> Message-ID: On Fri, Oct 11, 2013 at 8:35 AM, holger krekel wrote: > On Fri, Oct 11, 2013 at 08:25 -0300, Bruno Oliveira wrote: > > On Fri, Oct 11, 2013 at 8:14 AM, Vladimir Keleshev < > vladimir at keleshev.com>wrote: > > > > > > If you refer to tmpdir/monkeypatch and potentially others, i agree > > > > that the current behaviour is more surprising (sharing the tmpdir > > > > across multiple fixtures which don't even neccessarily know about > each > > > > other). > > > > > > Absolutely. Was a major WAT for me. > > > > > > > Weird, to me it was the exact opposite about tmpdir: it is a common usage > > in our code base to create a temporary directory for usage through a test > > suite (usually a module containing related tests). And different fixtures > > sharing tmpdir makes sense in this scenario. > > Are you really talking about the pytest fixture "tmpdir" here? > "Throughout the test suite" sounds like something else because > "tmpdir" is function scoped and created afresh for each test function > requesting it. > Sorry I wasn't very clear, I meant using the same fixture instance for each test, like two fixtures that depend on tmpdir (independently) and both fixtures are used in a test function that also needs a tmpdir; in this case the test method and fixtures all share the same tmpdir instance. But now I misunderstood the original issue, my apologies, now I see what was really meant. Cheers, Bruno. -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Fri Oct 11 16:12:16 2013 From: holger at merlinux.eu (holger krekel) Date: Fri, 11 Oct 2013 14:12:16 +0000 Subject: [pytest-dev] solving the "too static" fixture scope problem In-Reply-To: References: <20131011084045.GD14010@merlinux.eu> <3FD2A0D1-78B7-46AE-8B52-63958919032B@gmx.net> <20131011105724.GE14010@merlinux.eu> <293531381490040@web30m.yandex.ru> <20131011113524.GF14010@merlinux.eu> Message-ID: <20131011141216.GG14010@merlinux.eu> On Fri, Oct 11, 2013 at 10:51 -0300, Bruno Oliveira wrote: > On Fri, Oct 11, 2013 at 8:35 AM, holger krekel wrote: > > > On Fri, Oct 11, 2013 at 08:25 -0300, Bruno Oliveira wrote: > > > On Fri, Oct 11, 2013 at 8:14 AM, Vladimir Keleshev < > > vladimir at keleshev.com>wrote: > > > > > > > > If you refer to tmpdir/monkeypatch and potentially others, i agree > > > > > that the current behaviour is more surprising (sharing the tmpdir > > > > > across multiple fixtures which don't even neccessarily know about > > each > > > > > other). > > > > > > > > Absolutely. Was a major WAT for me. > > > > > > > > > > Weird, to me it was the exact opposite about tmpdir: it is a common usage > > > in our code base to create a temporary directory for usage through a test > > > suite (usually a module containing related tests). And different fixtures > > > sharing tmpdir makes sense in this scenario. > > > > Are you really talking about the pytest fixture "tmpdir" here? > > "Throughout the test suite" sounds like something else because > > "tmpdir" is function scoped and created afresh for each test function > > requesting it. > > > > Sorry I wasn't very clear, I meant using the same fixture instance for each > test, like two fixtures that depend on tmpdir (independently) and both > fixtures are used in a test function that also needs a tmpdir; in this case > the test method and fixtures all share the same tmpdir instance. Ok, and your setup expects to get the same tmpdir then, IIUC. I might even have a similar situtation myself in some projects. So i guess we will need to introduce a new name for the "each" scoped temporary dir. That is a minor issue (except that naming is hard, of course) compared to the actual implementation of the "each" scope. cheers, holger > But now I misunderstood the original issue, my apologies, now I see what > was really meant. > > Cheers, > Bruno. From lahwran0 at gmail.com Fri Oct 11 17:29:18 2013 From: lahwran0 at gmail.com (lahwran) Date: Fri, 11 Oct 2013 09:29:18 -0600 Subject: [pytest-dev] solving the "too static" fixture scope problem In-Reply-To: <20131011141216.GG14010@merlinux.eu> References: <20131011084045.GD14010@merlinux.eu> <3FD2A0D1-78B7-46AE-8B52-63958919032B@gmx.net> <20131011105724.GE14010@merlinux.eu> <293531381490040@web30m.yandex.ru> <20131011113524.GF14010@merlinux.eu> <20131011141216.GG14010@merlinux.eu> Message-ID: I've seen issues with the mock library where patching the same thing multiple times can cause the patchedness to leak out of a test - because the teardown order isn't guaranteed the way it was being used. So if multiple monkeypatches are going to be created at the same scope, the teardown order had better be reverse of construction, or you're going to risk leaking the outer monkeypatch. -- lahwran On Fri, Oct 11, 2013 at 8:12 AM, holger krekel wrote: > On Fri, Oct 11, 2013 at 10:51 -0300, Bruno Oliveira wrote: > > On Fri, Oct 11, 2013 at 8:35 AM, holger krekel > wrote: > > > > > On Fri, Oct 11, 2013 at 08:25 -0300, Bruno Oliveira wrote: > > > > On Fri, Oct 11, 2013 at 8:14 AM, Vladimir Keleshev < > > > vladimir at keleshev.com>wrote: > > > > > > > > > > If you refer to tmpdir/monkeypatch and potentially others, i > agree > > > > > > that the current behaviour is more surprising (sharing the tmpdir > > > > > > across multiple fixtures which don't even neccessarily know about > > > each > > > > > > other). > > > > > > > > > > Absolutely. Was a major WAT for me. > > > > > > > > > > > > > Weird, to me it was the exact opposite about tmpdir: it is a common > usage > > > > in our code base to create a temporary directory for usage through a > test > > > > suite (usually a module containing related tests). And different > fixtures > > > > sharing tmpdir makes sense in this scenario. > > > > > > Are you really talking about the pytest fixture "tmpdir" here? > > > "Throughout the test suite" sounds like something else because > > > "tmpdir" is function scoped and created afresh for each test function > > > requesting it. > > > > > > > Sorry I wasn't very clear, I meant using the same fixture instance for > each > > test, like two fixtures that depend on tmpdir (independently) and both > > fixtures are used in a test function that also needs a tmpdir; in this > case > > the test method and fixtures all share the same tmpdir instance. > > Ok, and your setup expects to get the same tmpdir then, IIUC. I might > even have a similar situtation myself in some projects. So i guess we > will need to introduce a new name for the "each" scoped temporary dir. > That is a minor issue (except that naming is hard, of course) compared > to the actual implementation of the "each" scope. > > cheers, > holger > > > But now I misunderstood the original issue, my apologies, now I see what > > was really meant. > > > > Cheers, > > Bruno. > _______________________________________________ > Pytest-dev mailing list > Pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronny.pfannschmidt at gmx.de Fri Oct 11 17:58:53 2013 From: ronny.pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Fri, 11 Oct 2013 17:58:53 +0200 Subject: [pytest-dev] Re: solving the "too static" fixture scope problem In-Reply-To: <20131011084045.GD14010@merlinux.eu> References: <20131011084045.GD14010@merlinux.eu> Message-ID: <1f8e0320-3257-4078-9c9a-e5be605d672a@gmx.de> Hi Holger, i think that just 'each' is too generic, i'd rather see something more specific like scope='session,function' i think its very important to be in control of the actual scopes used in a fixture an example i have in mind is a semi-generic tmpdir fixture @pytest.fixture(scope='session,function') def tmpdir(request, _pytest_basetmp): if request.scope == 'session': return _basetmp.ensure('session', dir=1) elif request.scope == 'function': return _basetmp.ensure('tests', dir=1)\ .make_numbered_dir(request.function__name__) or a little more detailed idea about databases: @pytest.fixture(scope='session,function'): def db_connection(request): if db_transactions_nested and request.scope=='session': conn = connect(...) schema.create_all(bind=conn) dbsetup.install_initial_data(bind=conn) return conn elif db_transactions_nested and request.scope=='function': conn = request.get_fixture('db_connection', scope='session') transaction = conn.begin_nested() request.addfinalizer(transaction.rollback) return conn elif request.scope=='function': conn = connect(...) schema.create_all(bind=conn) dbsetup.install_initial_data(bind=conn) return conn else: pyyest.fail('unexpected scope state') the key point for me is being explicit about scopes when declaring fixtures and requesting them. -- Ronny On Freitag, 11. Oktober 2013 10:40:45 CEST, holger krekel wrote: > Hi pytest users and developers, > > I'd like to discuss and get feedback on > extending the fixture mechanism and fix what > i consider a biggest limitation at the moment. > > Problem: fixtures are tied statically to a scope > ================================================= > > For example, you cannot use monkeypatch in a higher > than "function" scoped fixture. Same is true for > tmpdir and probably also many user-defined fixtures. > I've certainly had this problem myself many times > that i had a fixture function that didn't really > care in what scope it was used. There are > ways to get around this problem but they are not > pretty: > > @pytest.fixture(scope="module") > def myfix_module(request > return _myfix(request) > > @pytest.fixture(scope="function") > def myfix_function(request > return _myfix(request) > > where _myfix is the function that doesn't > care about the actual scope. Even then, you > can't use builtin fixtures like "monkeypatch", > "tmpdir", etc. > > Solution Idea: introduce "each" scoped fixtures > ===================================================== > > The idea is allow a fixture function to declare it wants > to be used in the scope of the requesting fixture function > (or at function-scope if used from a test). > > This is how "monkeypatch" would be implemented then: > > @pytest.fixture(scope="each") > def monkeypatch(request): > ... # same implementation as in _pytest/monkeypatch.py > > The new "each" scope means that each fixture/test requesting > the "monkeypatch" fixture would receive its own fixture instance. > > So a session-scoped fixture could naturally use it like this: > > @pytest.fixture(scope="session") > def myfix(monkeypatch): > monkeypatchsetattr(...) > return some_value > > The passed in monkeypatch object here is a specific instance just > for the ``myfix`` fixture function: "each" fixture function > requesting ``monkeypatch`` gets a new instance of it. > If e.g. a test uses another module-scoped fixture defined like this: > > @pytest.fixture(scope="module") > def myfix2(monkeypatch): > mp.setattr(...) > return some_value > > this would invoke the ``monkeypatch`` fixture function a second time, > resulting in a new instance for use by the ``myfix2`` instance. > > The same logic could be applied to other fixtures > like "tmpdir" or user-defined ones. > > Do you like this idea? Would you find it helpful for your test suites? > > There is one issue i am not sure about yet, however. Currently, > when a test requires fixture A and B, and B requires C and C requires A, > then the two "A" would be exactly the same object, independently of what > which scopes are declared. If A=="tmpdir", then the test's tmpdir and > C's tmpdir would be the same directory. I often don't find > this desirable. > If tmpdir would be an "each" scoped fixture, then C and the test would > each receive a clean new tmpdir. If that is a backward-compat issue, > we could introduce another name for the new "each" scoped tmpdir. > I usually find myself working around the problem of a "tmpdir" > shared by multiple different fixtures, though. > > cheers, > holger -- mfg, Ronny Pfannschmidt From florian.schulze at gmx.net Fri Oct 11 20:46:42 2013 From: florian.schulze at gmx.net (Florian Schulze) Date: Fri, 11 Oct 2013 20:46:42 +0200 Subject: [pytest-dev] solving the "too static" fixture scope problem In-Reply-To: <1f8e0320-3257-4078-9c9a-e5be605d672a@gmx.de> References: <20131011084045.GD14010@merlinux.eu> <1f8e0320-3257-4078-9c9a-e5be605d672a@gmx.de> Message-ID: On 11.10.2013, at 17:58, Ronny Pfannschmidt wrote: > Hi Holger, > > i think that just 'each' is too generic, > i'd rather see something more specific like scope='session,function' > > i think its very important to be in control of the actual scopes used in a fixture > > an example i have in mind is a semi-generic tmpdir fixture > > @pytest.fixture(scope='session,function') > def tmpdir(request, _pytest_basetmp): > if request.scope == 'session': > return _basetmp.ensure('session', dir=1) > elif request.scope == 'function': > return _basetmp.ensure('tests', dir=1)\ > .make_numbered_dir(request.function__name__) If something like this is chosen, then please use a list or tuple, not a comma separated string. Regards, Florian Schulze From holger at merlinux.eu Fri Oct 11 21:02:56 2013 From: holger at merlinux.eu (holger krekel) Date: Fri, 11 Oct 2013 19:02:56 +0000 Subject: [pytest-dev] solving the "too static" fixture scope problem In-Reply-To: References: <20131011084045.GD14010@merlinux.eu> <1f8e0320-3257-4078-9c9a-e5be605d672a@gmx.de> Message-ID: <20131011190256.GH14010@merlinux.eu> On Fri, Oct 11, 2013 at 20:46 +0200, Florian Schulze wrote: > On 11.10.2013, at 17:58, Ronny Pfannschmidt wrote: > > > Hi Holger, > > > > i think that just 'each' is too generic, > > i'd rather see something more specific like scope='session,function' > > > > i think its very important to be in control of the actual scopes used in a fixture > > > > an example i have in mind is a semi-generic tmpdir fixture > > > > @pytest.fixture(scope='session,function') > > def tmpdir(request, _pytest_basetmp): > > if request.scope == 'session': > > return _basetmp.ensure('session', dir=1) > > elif request.scope == 'function': > > return _basetmp.ensure('tests', dir=1)\ > > .make_numbered_dir(request.function__name__) > > If something like this is chosen, then please use a list or tuple, not a comma separated string. We already have precendence for this style. See "pytest.mark.parametrize" in http://pytest.org/latest/announce/release-2.4.0.html I find "session,function,module" nicer to type than ("session", "function", "module") and don't see an advantage to enforcing the latter. best, holger From ronny.pfannschmidt at gmx.de Fri Oct 11 21:25:29 2013 From: ronny.pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Fri, 11 Oct 2013 21:25:29 +0200 Subject: [pytest-dev] Re: solving the "too static" fixture scope problem In-Reply-To: <20131011190256.GH14010@merlinux.eu> References: <20131011084045.GD14010@merlinux.eu> <1f8e0320-3257-4078-9c9a-e5be605d672a@gmx.de> <20131011190256.GH14010@merlinux.eu> Message-ID: <2ada6cf0-a331-4eac-9350-110aafd0db86@gmx.de> as far as i can tell, we can just implement it the same way On Freitag, 11. Oktober 2013 21:02:56 CEST, holger krekel wrote: > On Fri, Oct 11, 2013 at 20:46 +0200, Florian Schulze wrote: >> On 11.10.2013, at 17:58, Ronny Pfannschmidt >> wrote: >> ... > > We already have precendence for this style. See > "pytest.mark.parametrize" in > http://pytest.org/latest/announce/release-2.4.0.html > > I find "session,function,module" nicer to type than > ("session", "function", "module") and don't see an > advantage to enforcing the latter. > > best, > holger > > -- mfg, Ronny Pfannschmidt From holger at merlinux.eu Sat Oct 12 15:12:36 2013 From: holger at merlinux.eu (holger krekel) Date: Sat, 12 Oct 2013 13:12:36 +0000 Subject: [pytest-dev] plugin status page Message-ID: <20131012131236.GK14010@merlinux.eu> Hi Bruno, all, i've brought your initial work online at the "dev" pytest pages: http://pytest.org/dev/plugins_index.html I think it's a good start, thanks! My comments and wishes (others may comment as well!): - i'd collapse NAME and VERSION columns to save space, i.e. "pytest-bdd-0.6.1". - we should try to collect repository locations. maybe parsing for github/bitbucket urls would yield most of them automatically? Maybe we need to add some manually. - what about adding download numbers? - as to code organisation: you can leave it as is for now or (maybe better) put all code and generated files into a dedicated directory. A very interesting bit will be the "2.4.2 compat" and "dev" compat determination. In fact, i think "devpi" should help with that although i guess i need to improve some things. You can give using it a try with these steps: pip install devpi-client devpi use http://devpi.net/root/pypi devpi test -e py27 pytest-pep8 The last bit should download the pypi-version of pytest-pep8, unpack it and run tox for the "py27" environment and then post back test results (as a json file) to the devpi.net server. You can also ask about plugin "test" status: devpi list pytest-pep8 I just found out that this doesn't work for "pytest-flakes" yet although that does have a tox.ini. need to look into what's going on. In principle, devpi-server (an instance of which you are using above), provides a lot of detailed test info as a json, see here for example: http://devpi.net/+tests/6199353734615fde47d1fbfef1ebc737/toxresult The test results are stored as per the package MD5 at the moment. The whole test/list/json machinery of devpi needs some more work to become really nice, but it should already help as currently implemented, maybe best by using "devpi list" and filing feature requests against it :) For testing "development" versions, it should be enough to upload e.g. a "pytest-2.4.3.dev1" to a private index and then re-run the script that invokes something like the above devpi commands for all of the plugins. Now, for the plugins that don't have a tox.ini we could supply one out of band. For this "devpi test" would grow an option like "--toxini=mytox.ini" and would use that instead of expecting one in the downloaded unpacked distribution file. This tox.ini would probably just have a testenv that does "py.test --version" and "py.test --help" to see if something broke by just installing the plugin. So much for now, holger From holger at merlinux.eu Sun Oct 13 10:53:02 2013 From: holger at merlinux.eu (holger krekel) Date: Sun, 13 Oct 2013 08:53:02 +0000 Subject: [pytest-dev] solving the "too static" fixture scope problem In-Reply-To: <1f8e0320-3257-4078-9c9a-e5be605d672a@gmx.de> References: <20131011084045.GD14010@merlinux.eu> <1f8e0320-3257-4078-9c9a-e5be605d672a@gmx.de> Message-ID: <20131013085302.GN14010@merlinux.eu> Hi Ronny, On Fri, Oct 11, 2013 at 17:58 +0200, Ronny Pfannschmidt wrote: > Hi Holger, > > i think that just 'each' is too generic, > i'd rather see something more specific like scope='session,function' TLDR, i think you are tackling a different issue and one which i am not sure we need to solve. > i think its very important to be in control of the actual scopes used in a fixture You already can, you just cannot specify a multitude of possible fixtures within one fixture function. > an example i have in mind is a semi-generic tmpdir fixture > > @pytest.fixture(scope='session,function') > def tmpdir(request, _pytest_basetmp): > if request.scope == 'session': > return _basetmp.ensure('session', dir=1) > elif request.scope == 'function': > return _basetmp.ensure('tests', dir=1)\ > .make_numbered_dir(request.function__name__) In my mail the use case was that each fixture/test receiving "mytmpdir" (working title for the new builtin fixture) gets a fresh new directory which it does _not_ share with any other fixture function. Such "helper" fixture functions are scope-agnostic, they don't care about which scope they are requested in, and will perform their finalization (if any) within the same scope as requested. By contrast, your example kind of extends the current API and thus one would expect that the "tmpdir" is shared on each scope. > or a little more detailed idea about databases: > > @pytest.fixture(scope='session,function'): > def db_connection(request): > if db_transactions_nested and request.scope=='session': > conn = connect(...) > schema.create_all(bind=conn) > dbsetup.install_initial_data(bind=conn) > return conn > elif db_transactions_nested and request.scope=='function': > conn = request.get_fixture('db_connection', scope='session') > transaction = conn.begin_nested() > request.addfinalizer(transaction.rollback) > return conn > elif request.scope=='function': > conn = connect(...) > schema.create_all(bind=conn) > dbsetup.install_initial_data(bind=conn) > return conn > else: > pyyest.fail('unexpected scope state') > the key point for me is being explicit about scopes when declaring fixtures and requesting them. This i would term a "nested resources" use case. You can already solve this today by writing something (slightly simplified) like this: @pytest.fixture(scope="session") def db_connection_session(request): conn = connect(...) schema.create_all(bind=conn) dbsetup.install_initial_data(bind=conn) return conn @pytest.fixture(scope="function") def db_connection(db_connection_session): transaction = conn.begin_nested() request.addfinalizer(transaction.rollback) return conn Which is very explicit. Users "db_connection_session" and "db_connection" know exactly what they are getting and the current fixture-scope-mismatch-detection machinery makes sure you don't use the wrong one in many cases. My original consideration really focuses around providing fixtures which are never cached and thus can remain ignorant on caching scope. Examples this is useful for: - mytmpdir (a fresh empty directory whenever requested) - monkeypatch (a fresh new instance/finalization at the same scope as the requestor) And a new one would be (i have this in several of my projects but with hacks): - Popen(...): subprocess Popen but making sure that all started subprocesses are terminated within the scope requested. E.g. if i use Popen in a module-scoped fixture, at the end of the module the subprocess will be killed. cheers, holger From vladimir at keleshev.com Mon Oct 14 17:19:12 2013 From: vladimir at keleshev.com (Vladimir Keleshev) Date: Mon, 14 Oct 2013 17:19:12 +0200 Subject: [pytest-dev] Allow pytest.raises to assert about exception arguments Message-ID: <11001381763952@web20h.yandex.ru> Right now one of my code-bases is filled with assertions like: with raises(SpecificError) as error: some_action() assert 'specific message' == str(error) Why not change `raises` to allow instances of exceptions as well: with raises(SpecificError('specific message')): some_action() I think, this can improve readability of `raises` assertions. Can't even think of immediate downsides of this. What do you think? ?Vladimir From nicoddemus at gmail.com Mon Oct 14 19:21:18 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Mon, 14 Oct 2013 14:21:18 -0300 Subject: [pytest-dev] plugin status page In-Reply-To: <20131012131236.GK14010@merlinux.eu> References: <20131012131236.GK14010@merlinux.eu> Message-ID: Hi Holgen, On Sat, Oct 12, 2013 at 10:12 AM, holger krekel wrote: > My comments and wishes (others may comment as well!): > > - i'd collapse NAME and VERSION columns to save space, i.e. > "pytest-bdd-0.6.1". > > - what about adding download numbers? > > - as to code organisation: you can leave it as is for now or (maybe > better) put all code and generated files into a dedicated directory No problem, will work on those today. :) > - we should try to collect repository locations. maybe parsing > for github/bitbucket urls would yield most of them automatically? > Maybe we need to add some manually. > May I ask why we would need the repository locations? I mean, to work on the compatibility feature we can work directly with packages in pypi or devpi... A very interesting bit will be the "2.4.2 compat" and "dev" compat > determination. In fact, i think "devpi" should help with that although > i guess i need to improve some things. You can give using it a try > with these steps: > > pip install devpi-client > devpi use http://devpi.net/root/pypi > devpi test -e py27 pytest-pep8 > > The last bit should download the pypi-version of pytest-pep8, unpack > it and run tox for the "py27" environment and then post back test > results (as a json file) to the devpi.net server. You can also ask about > plugin "test" status: > > devpi list pytest-pep8 > > I just found out that this doesn't work for "pytest-flakes" yet > although that does have a tox.ini. need to look into what's going on. > In principle, devpi-server (an instance of which you are using above), > provides a lot of detailed test info as a json, see here for example: > > http://devpi.net/+tests/6199353734615fde47d1fbfef1ebc737/toxresult > > The test results are stored as per the package MD5 at the moment. > The whole test/list/json machinery of devpi needs some more work > to become really nice, but it should already help as currently implemented, > maybe best by using "devpi list" and filing feature requests against it :) > > For testing "development" versions, it should be enough to upload > e.g. a "pytest-2.4.3.dev1" to a private index and then re-run the script > that invokes something like the above devpi commands for all of the > plugins. > > Now, for the plugins that don't have a tox.ini we could supply one > out of band. For this "devpi test" would grow an option like > "--toxini=mytox.ini" and would use that instead of expecting one > in the downloaded unpacked distribution file. This tox.ini would probably > just have a testenv that does "py.test --version" and "py.test --help" > to see if something broke by just installing the plugin. > Interesting! Will give it a try today. Will keep you posted as things move along. Cheers, Bruno. > So much for now, > holger > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Mon Oct 14 22:32:30 2013 From: holger at merlinux.eu (holger krekel) Date: Mon, 14 Oct 2013 20:32:30 +0000 Subject: [pytest-dev] plugin status page In-Reply-To: References: <20131012131236.GK14010@merlinux.eu> Message-ID: <20131014203230.GX14010@merlinux.eu> On Mon, Oct 14, 2013 at 14:21 -0300, Bruno Oliveira wrote: > Hi Holgen, hi Brunon! :) > > On Sat, Oct 12, 2013 at 10:12 AM, holger krekel wrote: > > > My comments and wishes (others may comment as well!): > > > > - i'd collapse NAME and VERSION columns to save space, i.e. > > "pytest-bdd-0.6.1". > > > > - what about adding download numbers? > > > > - as to code organisation: you can leave it as is for now or (maybe > > better) put all code and generated files into a dedicated directory > > > No problem, will work on those today. :) > > > > - we should try to collect repository locations. maybe parsing > > for github/bitbucket urls would yield most of them automatically? > > Maybe we need to add some manually. > > > > May I ask why we would need the repository locations? I mean, to work on > the compatibility feature we can work directly with packages in pypi or > devpi... purely for the person reading the page and wanting to look at the code or do a PR. > A very interesting bit will be the "2.4.2 compat" and "dev" compat > > determination. In fact, i think "devpi" should help with that although > > i guess i need to improve some things. You can give using it a try > > with these steps: > > > > pip install devpi-client > > devpi use http://devpi.net/root/pypi > > devpi test -e py27 pytest-pep8 > > > > The last bit should download the pypi-version of pytest-pep8, unpack > > it and run tox for the "py27" environment and then post back test > > results (as a json file) to the devpi.net server. You can also ask about > > plugin "test" status: > > > > devpi list pytest-pep8 > > > > I just found out that this doesn't work for "pytest-flakes" yet > > although that does have a tox.ini. need to look into what's going on. > > In principle, devpi-server (an instance of which you are using above), > > provides a lot of detailed test info as a json, see here for example: > > > > http://devpi.net/+tests/6199353734615fde47d1fbfef1ebc737/toxresult > > > > The test results are stored as per the package MD5 at the moment. > > The whole test/list/json machinery of devpi needs some more work > > to become really nice, but it should already help as currently implemented, > > maybe best by using "devpi list" and filing feature requests against it :) > > > > For testing "development" versions, it should be enough to upload > > e.g. a "pytest-2.4.3.dev1" to a private index and then re-run the script > > that invokes something like the above devpi commands for all of the > > plugins. > > > > Now, for the plugins that don't have a tox.ini we could supply one > > out of band. For this "devpi test" would grow an option like > > "--toxini=mytox.ini" and would use that instead of expecting one > > in the downloaded unpacked distribution file. This tox.ini would probably > > just have a testenv that does "py.test --version" and "py.test --help" > > to see if something broke by just installing the plugin. > > > > Interesting! Will give it a try today. Will keep you posted as things move > along. great, i am travelling to Pycon DE tomorrow morning but should be online from time to time. best, holger > Cheers, > Bruno. > > > > So much for now, > > holger > > > > From holger at merlinux.eu Mon Oct 14 22:50:24 2013 From: holger at merlinux.eu (holger krekel) Date: Mon, 14 Oct 2013 20:50:24 +0000 Subject: [pytest-dev] Allow pytest.raises to assert about exception arguments In-Reply-To: <11001381763952@web20h.yandex.ru> References: <11001381763952@web20h.yandex.ru> Message-ID: <20131014205024.GA14010@merlinux.eu> On Mon, Oct 14, 2013 at 17:19 +0200, Vladimir Keleshev wrote: > Right now one of my code-bases is filled with assertions like: > > with raises(SpecificError) as error: > some_action() > assert 'specific message' == str(error) > > Why not change `raises` to allow instances of exceptions as well: > > with raises(SpecificError('specific message')): > some_action() This depends on defining equality on the exception object, i guess. For example: >>> KeyError("hello") == KeyError("hello") False So it doesn't seem to be defined by default. Of course we could do something ourselves but i am not sure how reliable that would be. Experiment if you like :) holger > I think, this can improve readability of `raises` assertions. Can't even think of immediate downsides of this. > > What do you think? > > ?Vladimir > _______________________________________________ > Pytest-dev mailing list > Pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev From nicoddemus at gmail.com Mon Oct 14 23:19:59 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Mon, 14 Oct 2013 18:19:59 -0300 Subject: [pytest-dev] Allow pytest.raises to assert about exception arguments In-Reply-To: <20131014205024.GA14010@merlinux.eu> References: <11001381763952@web20h.yandex.ru> <20131014205024.GA14010@merlinux.eu> Message-ID: An alternative would be to use another "raises" function, like with raises_message(SpecificError, 'specific message'): some_action() One problem is bloating the API a little more... Cheers, Bruno. On Mon, Oct 14, 2013 at 5:50 PM, holger krekel wrote: > On Mon, Oct 14, 2013 at 17:19 +0200, Vladimir Keleshev wrote: > > Right now one of my code-bases is filled with assertions like: > > > > with raises(SpecificError) as error: > > some_action() > > assert 'specific message' == str(error) > > > > Why not change `raises` to allow instances of exceptions as well: > > > > with raises(SpecificError('specific message')): > > some_action() > > This depends on defining equality on the exception object, i guess. > For example: > > >>> KeyError("hello") == KeyError("hello") > False > > So it doesn't seem to be defined by default. Of course we could do > something ourselves but i am not sure how reliable that would be. > Experiment if you like :) > > holger > > > > I think, this can improve readability of `raises` assertions. Can't even > think of immediate downsides of this. > > > > What do you think? > > > > ?Vladimir > > _______________________________________________ > > Pytest-dev mailing list > > Pytest-dev at python.org > > https://mail.python.org/mailman/listinfo/pytest-dev > _______________________________________________ > Pytest-dev mailing list > Pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicoddemus at gmail.com Tue Oct 15 04:52:11 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Mon, 14 Oct 2013 23:52:11 -0300 Subject: [pytest-dev] plugin status page In-Reply-To: <20131014203230.GX14010@merlinux.eu> References: <20131012131236.GK14010@merlinux.eu> <20131014203230.GX14010@merlinux.eu> Message-ID: Hi Holger, On Mon, Oct 14, 2013 at 5:32 PM, holger krekel wrote: > On Mon, Oct 14, 2013 at 14:21 -0300, Bruno Oliveira wrote: > > Hi Holgen, > > hi Brunon! :) > Hehehe, oops. > > > On Sat, Oct 12, 2013 at 10:12 AM, holger krekel > wrote: > > > > > > - i'd collapse NAME and VERSION columns to save space, i.e. > > > "pytest-bdd-0.6.1". > > > > > > - what about adding download numbers? > > > > > > - as to code organisation: you can leave it as is for now or (maybe > > > better) put all code and generated files into a dedicated directory > > > Done, opened a PR for discussion and reviewing. :) > > > > > > - we should try to collect repository locations. maybe parsing > > > for github/bitbucket urls would yield most of them automatically? > > > Maybe we need to add some manually. > > > > > > > May I ask why we would need the repository locations? I mean, to work on > > the compatibility feature we can work directly with packages in pypi or > > devpi... > > purely for the person reading the page and wanting to look at the code > or do a PR. > Oh I see! Well, parsing the urls seems problematic, because we also have to take in account the username ("hpk42/pytest" for example). I looked at the GitHub and BitBucket's rest APIs to see if we could use them for searching, but it seems only GitHub at the moment has a search API (and in beta stage at that). I think we will have to maintain the repository list manually. Package authors can help as well by simply opening PRs adding repositories for their plugins I guess. > A very interesting bit will be the "2.4.2 compat" and "dev" compat > > > Some late night quick hacking and I have come up with this proof of concept: https://www.travis-ci.org/nicoddemus/pytest-plugs As it is right now it serves only as a prove of concept, of course. The idea is using travis to handle running things for us, driving it using a script that downloads and run tests using tox. We can then collect json information from that and POST those results back to devpi. Also we can use its build matrix to test against different pytest versions, for example. I guess we can implement in devpi a concept similar to travis build status imagens, which we could then use in the plugins_index page. Against python 2.6 the script is failing, I have yet to investigate. great, i am travelling to Pycon DE tomorrow morning but should be online > from time to time. > Nice tripe and good PyCon! Cheers, Bruno. -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Sun Oct 20 07:52:19 2013 From: holger at merlinux.eu (holger krekel) Date: Sun, 20 Oct 2013 05:52:19 +0000 Subject: [pytest-dev] plugin status page In-Reply-To: References: <20131012131236.GK14010@merlinux.eu> <20131014203230.GX14010@merlinux.eu> Message-ID: <20131020055219.GB19953@merlinux.eu> On Mon, Oct 14, 2013 at 23:52 -0300, Bruno Oliveira wrote: > > > > > > > > - i'd collapse NAME and VERSION columns to save space, i.e. > > > > "pytest-bdd-0.6.1". > > > > > > > > - what about adding download numbers? > > > > > > > > - as to code organisation: you can leave it as is for now or (maybe > > > > better) put all code and generated files into a dedicated directory > > > > > > > Done, opened a PR for discussion and reviewing. :) only got to it now, looks good so far! > > > > - we should try to collect repository locations. maybe parsing > > > > for github/bitbucket urls would yield most of them automatically? > > > > Maybe we need to add some manually. > > > > > > > > > > May I ask why we would need the repository locations? I mean, to work on > > > the compatibility feature we can work directly with packages in pypi or > > > devpi... > > > > purely for the person reading the page and wanting to look at the code > > or do a PR. > > > > Oh I see! Well, parsing the urls seems problematic, because we also have to > take in account the username ("hpk42/pytest" for example). I looked at the > GitHub and BitBucket's rest APIs to see if we could use them for searching, > but it seems only GitHub at the moment has a search API (and in beta stage > at that). > > I think we will have to maintain the repository list manually. Package > authors can help as well by simply opening PRs adding repositories for > their plugins I guess. What about generating a "plugin details " page for each plugin so we don't have to think too hard about what to put in the index page? > > A very interesting bit will be the "2.4.2 compat" and "dev" compat > > > > > > > Some late night quick hacking and I have come up with this proof of concept: > > https://www.travis-ci.org/nicoddemus/pytest-plugs > > As it is right now it serves only as a prove of concept, of course. The > idea is using travis to handle running things for us, driving it using a > script that downloads and run tests using tox. We can then collect json > information from that and POST those results back to devpi. Also we can use > its build matrix to test against different pytest versions, for example. Hum, interesting. Is this mainly to make use of travis-ci's build hosts? What is the advantage over just invoking "devpi test PLUGIN-NAME" from the script to automate the download/unpack/test/post-results steps? > I guess we can implement in devpi a concept similar to travis build status > imagens, which we could then use in the plugins_index page. > > Against python 2.6 the script is failing, I have yet to investigate. > > great, i am travelling to Pycon DE tomorrow morning but should be online > > from time to time. > > > > Nice tripe and good PyCon! Thanks! Got back but caught a flue ... cheers, holger From florian.schulze at gmx.net Sun Oct 20 10:31:06 2013 From: florian.schulze at gmx.net (Florian Schulze) Date: Sun, 20 Oct 2013 10:31:06 +0200 Subject: [pytest-dev] plugin status page In-Reply-To: <20131020055219.GB19953@merlinux.eu> References: <20131012131236.GK14010@merlinux.eu> <20131014203230.GX14010@merlinux.eu> <20131020055219.GB19953@merlinux.eu> Message-ID: <708C78B9-CCD7-4A72-BDAB-8A42CA2A87BB@gmx.net> >>> A very interesting bit will be the "2.4.2 compat" and "dev" compat >>>> >>> >> >> Some late night quick hacking and I have come up with this proof of concept: >> >> https://www.travis-ci.org/nicoddemus/pytest-plugs >> >> As it is right now it serves only as a prove of concept, of course. The >> idea is using travis to handle running things for us, driving it using a >> script that downloads and run tests using tox. We can then collect json >> information from that and POST those results back to devpi. Also we can use >> its build matrix to test against different pytest versions, for example. > > Hum, interesting. Is this mainly to make use of travis-ci's build hosts? > What is the advantage over just invoking "devpi test PLUGIN-NAME" from the > script to automate the download/unpack/test/post-results steps? I'd say the biggest advantage is to not execute stuff on your own server. They have to deal with that anyway. Regards, Florian Schulze From nicoddemus at gmail.com Sun Oct 20 15:06:37 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Sun, 20 Oct 2013 11:06:37 -0200 Subject: [pytest-dev] plugin status page In-Reply-To: <20131020055219.GB19953@merlinux.eu> References: <20131012131236.GK14010@merlinux.eu> <20131014203230.GX14010@merlinux.eu> <20131020055219.GB19953@merlinux.eu> Message-ID: On Sun, Oct 20, 2013 at 3:52 AM, holger krekel wrote: > > Done, opened a PR for discussion and reviewing. :) > > only got to it now, looks good so far! > Bummer, I thought merging it would automatically resolve the issue I created (as in GitHub)... oh well. :) > I think we will have to maintain the repository list manually. Package > > authors can help as well by simply opening PRs adding repositories for > > their plugins I guess. > > What about generating a "plugin details " page for each plugin so we don't > have to think too hard about what to put in the index page? > That's a possibility, for sure. > > > A very interesting bit will be the "2.4.2 compat" and "dev" compat > > > > > > > > > > > Some late night quick hacking and I have come up with this proof of > concept: > > > > https://www.travis-ci.org/nicoddemus/pytest-plugs > > > > As it is right now it serves only as a prove of concept, of course. The > > idea is using travis to handle running things for us, driving it using a > > script that downloads and run tests using tox. We can then collect json > > information from that and POST those results back to devpi. Also we can > use > > its build matrix to test against different pytest versions, for example. > > Hum, interesting. Is this mainly to make use of travis-ci's build hosts? > As Florian said, mainly so you don't need to have a dedicated host to run those scripts yourself. > What is the advantage over just invoking "devpi test PLUGIN-NAME" from the > script to automate the download/unpack/test/post-results steps? > I didn't use devpi at that point because of my limited knowledge of devpi at the moment, but that is certainly the next step. I will work on that next, and also work on generating a default tox.ini file for plugins that don't have one as you suggested. > > I guess we can implement in devpi a concept similar to travis build > status > > imagens, which we could then use in the plugins_index page. > > > > Against python 2.6 the script is failing, I have yet to investigate. > I found the reason for that (http vs https) and now the job runs against Python 2.6, 2.7, 3.2 and 3.2. Thanks! Got back but caught a flue ... > It happens, hope you get better soon. :) Cheers, Bruno. On Sun, Oct 20, 2013 at 3:52 AM, holger krekel wrote: > On Mon, Oct 14, 2013 at 23:52 -0300, Bruno Oliveira wrote: > > > > > > > > > > - i'd collapse NAME and VERSION columns to save space, i.e. > > > > > "pytest-bdd-0.6.1". > > > > > > > > > > - what about adding download numbers? > > > > > > > > > > - as to code organisation: you can leave it as is for now or (maybe > > > > > better) put all code and generated files into a dedicated > directory > > > > > > > > > > > Done, opened a PR for discussion and reviewing. :) > > only got to it now, looks good so far! > > > > > > - we should try to collect repository locations. maybe parsing > > > > > for github/bitbucket urls would yield most of them automatically? > > > > > Maybe we need to add some manually. > > > > > > > > > > > > > May I ask why we would need the repository locations? I mean, to > work on > > > > the compatibility feature we can work directly with packages in pypi > or > > > > devpi... > > > > > > purely for the person reading the page and wanting to look at the code > > > or do a PR. > > > > > > > Oh I see! Well, parsing the urls seems problematic, because we also have > to > > take in account the username ("hpk42/pytest" for example). I looked at > the > > GitHub and BitBucket's rest APIs to see if we could use them for > searching, > > but it seems only GitHub at the moment has a search API (and in beta > stage > > at that). > > > > I think we will have to maintain the repository list manually. Package > > authors can help as well by simply opening PRs adding repositories for > > their plugins I guess. > > What about generating a "plugin details " page for each plugin so we don't > have to think too hard about what to put in the index page? > > > > A very interesting bit will be the "2.4.2 compat" and "dev" compat > > > > > > > > > > > Some late night quick hacking and I have come up with this proof of > concept: > > > > https://www.travis-ci.org/nicoddemus/pytest-plugs > > > > As it is right now it serves only as a prove of concept, of course. The > > idea is using travis to handle running things for us, driving it using a > > script that downloads and run tests using tox. We can then collect json > > information from that and POST those results back to devpi. Also we can > use > > its build matrix to test against different pytest versions, for example. > > Hum, interesting. Is this mainly to make use of travis-ci's build hosts? > What is the advantage over just invoking "devpi test PLUGIN-NAME" from the > script to automate the download/unpack/test/post-results steps? > > > I guess we can implement in devpi a concept similar to travis build > status > > imagens, which we could then use in the plugins_index page. > > > > Against python 2.6 the script is failing, I have yet to investigate. > > > > great, i am travelling to Pycon DE tomorrow morning but should be online > > > from time to time. > > > > > > > Nice tripe and good PyCon! > > Thanks! Got back but caught a flue ... > > cheers, > holger > -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Mon Oct 21 14:12:46 2013 From: holger at merlinux.eu (holger krekel) Date: Mon, 21 Oct 2013 12:12:46 +0000 Subject: [pytest-dev] parametrization ids simplification/generalization Message-ID: <20131021121246.GE3973@merlinux.eu> Hi Brianna, all, I am thinking about constructing ids for parametrized tests using argname/valindex, i.e. not use the actual parametrization values. The latter causes some problems (among them also issue357) and generally doesn't allow to distinguish the two tests in: @pytest.mark.parametrize("arg", [1,1]) def test_some(arg): pass Currently this gives two identical test ids "test_some[1]". With the new scheme this gives: test_some[arg0] test_some[arg1] Such a scheme would guarantee unique IDs. The scheme is already used for user instances. Note that when you get a traceback you would still see the actual argument values in the traceback. Any objections? holger From brianna.laugher at gmail.com Tue Oct 22 02:57:07 2013 From: brianna.laugher at gmail.com (Brianna Laugher) Date: Tue, 22 Oct 2013 11:57:07 +1100 Subject: [pytest-dev] parametrization ids simplification/generalization In-Reply-To: <20131021121246.GE3973@merlinux.eu> References: <20131021121246.GE3973@merlinux.eu> Message-ID: Hi, I don't mind losing the IDs constructed from values, assuming it will still be possible to specify IDs if using metafunc.addcall. But if we are going to lose the IDs from values then they should probably just be plain integers (as happens when using metafunc.addall without specifying id) - having names like test_some[arg0-arg1-arg2], test_some[arg3-arg4-arg5] etc is totally pointless. thanks Brianna On 21 October 2013 23:12, holger krekel wrote: > Hi Brianna, all, > > I am thinking about constructing ids for parametrized tests using > argname/valindex, i.e. not use the actual parametrization > values. The latter causes some problems (among them also issue357) > and generally doesn't allow to distinguish the two tests in: > > @pytest.mark.parametrize("arg", [1,1]) > def test_some(arg): > pass > > Currently this gives two identical test ids "test_some[1]". > With the new scheme this gives: > > test_some[arg0] > test_some[arg1] > > Such a scheme would guarantee unique IDs. The scheme is already > used for user instances. > > Note that when you get a traceback you would still see the actual > argument values in the traceback. > > Any objections? > > holger > -- They've just been waiting in a mountain for the right moment: http://modernthings.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Tue Oct 22 09:59:02 2013 From: holger at merlinux.eu (holger krekel) Date: Tue, 22 Oct 2013 07:59:02 +0000 Subject: [pytest-dev] parametrization ids simplification/generalization In-Reply-To: References: <20131021121246.GE3973@merlinux.eu> Message-ID: <20131022075902.GI3973@merlinux.eu> On Tue, Oct 22, 2013 at 11:57 +1100, Brianna Laugher wrote: > I don't mind losing the IDs constructed from values, assuming it will still > be possible to specify IDs if using metafunc.addcall. But if we are going > to lose the IDs from values then they should probably just be plain > integers (as happens when using metafunc.addall without specifying id) - > having names like test_some[arg0-arg1-arg2], test_some[arg3-arg4-arg5] etc > is totally pointless. Including the argnames gives more context, no? Something like test_some[0-1-2] gives less. More importantly, selecting a specifically parametrized test works with "py.test -k arg0" whereas "py.test -k 1" would not work, i am afraid. best, holger > thanks > Brianna > > > > On 21 October 2013 23:12, holger krekel wrote: > > > Hi Brianna, all, > > > > I am thinking about constructing ids for parametrized tests using > > argname/valindex, i.e. not use the actual parametrization > > values. The latter causes some problems (among them also issue357) > > and generally doesn't allow to distinguish the two tests in: > > > > @pytest.mark.parametrize("arg", [1,1]) > > def test_some(arg): > > pass > > > > Currently this gives two identical test ids "test_some[1]". > > With the new scheme this gives: > > > > test_some[arg0] > > test_some[arg1] > > > > Such a scheme would guarantee unique IDs. The scheme is already > > used for user instances. > > > > Note that when you get a traceback you would still see the actual > > argument values in the traceback. > > > > Any objections? > > > > holger > > > > > > -- > They've just been waiting in a mountain for the right moment: > http://modernthings.org/ From brianna.laugher at gmail.com Tue Oct 22 10:17:02 2013 From: brianna.laugher at gmail.com (Brianna Laugher) Date: Tue, 22 Oct 2013 19:17:02 +1100 Subject: [pytest-dev] parametrization ids simplification/generalization In-Reply-To: <20131022075902.GI3973@merlinux.eu> References: <20131021121246.GE3973@merlinux.eu> <20131022075902.GI3973@merlinux.eu> Message-ID: So if you had this: @py.test.mark.parametrize(('a', 'b', 'expected'), [ (1, 1, 2), (1, 0, 1), (1, -1, 0), ]) def test_addition(a, b, expected): assert a + b == expected what would the generated IDs be? test_addition[a0-b0-expected0] test_addition[a1-b1-expected1] test_addition[a2-b2-expected2] That just seems really pointless to me. If you're not going to display the values at all, why bother including each parameter's name? I was thinking test_addition[0] test_addition[1] test_addition[2] (again, as it already is if you parametrize using metafunc.addcall), although I accept that is rather brief, maybe better as something like test_addition[param0] test_addition[param1] test_addition[param2] but basically I don't see the benefit of replicating how many parametrized values there are, in each test id. "-k arg0" is still going to be a pretty useless specification, just think if you have multiple parametrized tests in a single file. I think whatever solution here is not going to help the fact that you cannot specify an entire test name as in "-k 'test_addition[param0]'" - which I think will be fixed separately, as we discussed on issue #357. cheers, Brianna On 22 October 2013 18:59, holger krekel wrote: > On Tue, Oct 22, 2013 at 11:57 +1100, Brianna Laugher wrote: > > I don't mind losing the IDs constructed from values, assuming it will > still > > be possible to specify IDs if using metafunc.addcall. But if we are going > > to lose the IDs from values then they should probably just be plain > > integers (as happens when using metafunc.addall without specifying id) - > > having names like test_some[arg0-arg1-arg2], test_some[arg3-arg4-arg5] > etc > > is totally pointless. > > Including the argnames gives more context, no? Something like > test_some[0-1-2] gives less. More importantly, selecting a specifically > parametrized test works with "py.test -k arg0" whereas "py.test -k 1" > would not work, i am afraid. > > best, > holger > > > thanks > > Brianna > > > > > > > > On 21 October 2013 23:12, holger krekel wrote: > > > > > Hi Brianna, all, > > > > > > I am thinking about constructing ids for parametrized tests using > > > argname/valindex, i.e. not use the actual parametrization > > > values. The latter causes some problems (among them also issue357) > > > and generally doesn't allow to distinguish the two tests in: > > > > > > @pytest.mark.parametrize("arg", [1,1]) > > > def test_some(arg): > > > pass > > > > > > Currently this gives two identical test ids "test_some[1]". > > > With the new scheme this gives: > > > > > > test_some[arg0] > > > test_some[arg1] > > > > > > Such a scheme would guarantee unique IDs. The scheme is already > > > used for user instances. > > > > > > Note that when you get a traceback you would still see the actual > > > argument values in the traceback. > > > > > > Any objections? > > > > > > holger > > > > > > > > > > > -- > > They've just been waiting in a mountain for the right moment: > > http://modernthings.org/ > -- They've just been waiting in a mountain for the right moment: http://modernthings.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Tue Oct 22 10:40:01 2013 From: holger at merlinux.eu (holger krekel) Date: Tue, 22 Oct 2013 08:40:01 +0000 Subject: [pytest-dev] parametrization ids simplification/generalization In-Reply-To: References: <20131021121246.GE3973@merlinux.eu> <20131022075902.GI3973@merlinux.eu> Message-ID: <20131022084001.GJ3973@merlinux.eu> On Tue, Oct 22, 2013 at 19:17 +1100, Brianna Laugher wrote: > So if you had this: > > @py.test.mark.parametrize(('a', 'b', 'expected'), [ > (1, 1, 2), > (1, 0, 1), > (1, -1, 0), > ]) > def test_addition(a, b, expected): > assert a + b == expected > > what would the generated IDs be? > > test_addition[a0-b0-expected0] > test_addition[a1-b1-expected1] > test_addition[a2-b2-expected2] Yes. > That just seems really pointless to me. If you're not going to display the > values at all, why bother including each parameter's name? > > I was thinking > > test_addition[0] > test_addition[1] > test_addition[2] That's an option i guess. However, i'd rather like to see argument names, because there are other ways to paraemtrize and they combine, i.e. at the fixture function. If you have: @pytest.fixture(params=[1,2,3]) def arg(request): return request.param @pytest.mark.parametrize("otherarg", ["a", "b"]) def test_func(otherarg, arg): ... then test ids like: test_func[0] test_func[1] test_func[2] test_func[3] test_func[4] test_func[5] would not really give much of a clue whereas these ids: test_func[otherarg0-arg0] test_func[otherarg1-arg0] ... would give more of a clue, wouldn't they? > (again, as it already is if you parametrize using metafunc.addcall), > although I accept that is rather brief, maybe better as something like > > test_addition[param0] > test_addition[param1] > test_addition[param2] > > but basically I don't see the benefit of replicating how many parametrized > values there are, in each test id. > > "-k arg0" is still going to be a pretty useless specification, just think > if you have multiple parametrized tests in a single file. I think whatever > solution here is not going to help the fact that you cannot specify an > entire test name as in "-k 'test_addition[param0]'" - which I think will be > fixed separately, as we discussed on issue #357. Are all of your tests going to use "arg" as the argument name? I usually use less generic names and thus e. g. "-k http_address0" is pretty specific. That aside, are you aware you can always specify a testid path without using -k? For example: py.test test_module::test_func[otherarg0-arg0] You get such IDs by specifying "-rf" which lists IDs of failing tests at the end and we could add "-r." to mean to also show IDs for passing tests as well. best, holger From daniel.nouri at gmail.com Wed Oct 23 19:30:31 2013 From: daniel.nouri at gmail.com (Daniel Nouri) Date: Wed, 23 Oct 2013 19:30:31 +0200 Subject: [pytest-dev] monkeypatch.setattr of functions and from imports Message-ID: <526807B7.4070900@gmail.com> Dear all I have a function 'somefunc' in module 'a'. Another module 'b' imports that function with a "from import": a.py: def somefunc(): return 'hey!' b.py: from a import somefunc def someotherfunc(): return somefunc() + 'there' I want to now test 'someotherfunc' while patching away 'somefunc': def test_someotherfunc(monkeypatch): from b import someotherfunc monkeypatch.setattr('a.somefunc', lambda: 'eh?') someotherfunc() But this will fail, since 'b' imported somefunc from 'a' before we monkey patched the module's attribute. Without a "from import", this would work: b.py: import a def someotherfunc(): return a.somefunc() + 'there' What I would normally resort to is patch somefunc's func_code, so that even code that used a "from import" before I could patch will use the patched version. Thoughts? Cheers Daniel From holger at merlinux.eu Wed Oct 23 20:36:14 2013 From: holger at merlinux.eu (holger krekel) Date: Wed, 23 Oct 2013 18:36:14 +0000 Subject: [pytest-dev] monkeypatch.setattr of functions and from imports In-Reply-To: <526807B7.4070900@gmail.com> References: <526807B7.4070900@gmail.com> Message-ID: <20131023183613.GS3973@merlinux.eu> Hi Daniel, On Wed, Oct 23, 2013 at 19:30 +0200, Daniel Nouri wrote: > Dear all > > I have a function 'somefunc' in module 'a'. Another module 'b' imports > that function with a "from import": > > a.py: > > def somefunc(): > return 'hey!' > > b.py: > > from a import somefunc > > def someotherfunc(): > return somefunc() + 'there' > > > I want to now test 'someotherfunc' while patching away 'somefunc': > > def test_someotherfunc(monkeypatch): > from b import someotherfunc > monkeypatch.setattr('a.somefunc', lambda: 'eh?') > someotherfunc() > > But this will fail, since 'b' imported somefunc from 'a' before we > monkey patched the module's attribute. Without a "from import", this > would work: > > b.py: > > import a > > def someotherfunc(): > return a.somefunc() + 'there' > > What I would normally resort to is patch somefunc's func_code, so that > even code that used a "from import" before I could patch will use the > patched version. I would probably patch b's somefunc. Note that setattr() will by default raise an exception if "b.somefunc" does not exist. As to patching func_code: good idea, i had forgotten about assigning func_code. In earlier Python2.X versions assigning to func_code didn't work but i just checked that it does on py27 and py33. So it's definitely worthwhile to pursue. In your example you could do: monkeypatch.setattr('a.somefunc.func_code', (lambda: 'eh?').func_code) and it works (needs to use __code__ on py3). I guess we could think about a monkeypatch.setcode("a.somefunc", lambda: 'eh?') helper. You could either pass in a function or a code object. We could even think about allowing non-string targets: monkeypatch.setcode(a.somefunc, lambda: 'eh?') Or would "monkeypatch.setfunc" be a better name? best, holger From carl at oddbird.net Wed Oct 23 20:41:23 2013 From: carl at oddbird.net (Carl Meyer) Date: Wed, 23 Oct 2013 12:41:23 -0600 Subject: [pytest-dev] monkeypatch.setattr of functions and from imports In-Reply-To: <526816EF.9020303@oddbird.net> References: <526807B7.4070900@gmail.com> <526816EF.9020303@oddbird.net> Message-ID: <52681853.1010201@oddbird.net> On 10/23/2013 12:35 PM, Carl Meyer wrote: > On 10/23/2013 11:30 AM, Daniel Nouri wrote: >> def test_someotherfunc(monkeypatch): >> from b import someotherfunc >> monkeypatch.setattr('a.somefunc', lambda: 'eh?') >> someotherfunc() > > This doesn't look like correct syntax for monkeypatch.setattr. I think > you mean: > > monkeypatch.setattr(a, 'somefunc', lambda: 'eh?') Never mind this! I checked in pytest 2.3.5 on the project I had handy, didn't realize this syntax had been added in pytest 2.4. Nice! Carl From carl at oddbird.net Wed Oct 23 20:35:27 2013 From: carl at oddbird.net (Carl Meyer) Date: Wed, 23 Oct 2013 12:35:27 -0600 Subject: [pytest-dev] monkeypatch.setattr of functions and from imports In-Reply-To: <526807B7.4070900@gmail.com> References: <526807B7.4070900@gmail.com> Message-ID: <526816EF.9020303@oddbird.net> Hi Daniel, On 10/23/2013 11:30 AM, Daniel Nouri wrote: > I have a function 'somefunc' in module 'a'. Another module 'b' imports > that function with a "from import": > > a.py: > > def somefunc(): > return 'hey!' > > b.py: > > from a import somefunc > > def someotherfunc(): > return somefunc() + 'there' > > > I want to now test 'someotherfunc' while patching away 'somefunc': > > def test_someotherfunc(monkeypatch): > from b import someotherfunc > monkeypatch.setattr('a.somefunc', lambda: 'eh?') > someotherfunc() This doesn't look like correct syntax for monkeypatch.setattr. I think you mean: monkeypatch.setattr(a, 'somefunc', lambda: 'eh?') > But this will fail, since 'b' imported somefunc from 'a' before we > monkey patched the module's attribute. Without a "from import", this > would work: > > b.py: > > import a > > def someotherfunc(): > return a.somefunc() + 'there' > > What I would normally resort to is patch somefunc's func_code, so that > even code that used a "from import" before I could patch will use the > patched version. > > Thoughts? The simpler approach is to patch in the module where the function is used, not the module where it is defined. This has the added advantage of limiting the scope of your monkeypatching to only the module where you actually need it. So monkeypatch b.somefunc instead of a.somefunc: monkeypatch.setattr(b, 'somefunc', lambda: 'eh?') The key point here is to think of monkeypatching as just "reassign some name in some namespace", which makes it natural to reassign the name within the namespace where your code-under-test is actually using the name. Carl From daniel.nouri at gmail.com Thu Oct 24 00:33:41 2013 From: daniel.nouri at gmail.com (Daniel Nouri) Date: Thu, 24 Oct 2013 00:33:41 +0200 Subject: [pytest-dev] monkeypatch.setattr of functions and from imports In-Reply-To: <20131023183613.GS3973@merlinux.eu> References: <526807B7.4070900@gmail.com> <20131023183613.GS3973@merlinux.eu> Message-ID: <52684EC5.2000101@gmail.com> On 10/23/2013 08:36 PM, holger krekel wrote: > Hi Daniel, > > On Wed, Oct 23, 2013 at 19:30 +0200, Daniel Nouri wrote: >> Dear all >> >> I have a function 'somefunc' in module 'a'. Another module 'b' imports >> that function with a "from import": >> >> a.py: >> >> def somefunc(): >> return 'hey!' >> >> b.py: >> >> from a import somefunc >> >> def someotherfunc(): >> return somefunc() + 'there' >> >> >> I want to now test 'someotherfunc' while patching away 'somefunc': >> >> def test_someotherfunc(monkeypatch): >> from b import someotherfunc >> monkeypatch.setattr('a.somefunc', lambda: 'eh?') >> someotherfunc() >> >> But this will fail, since 'b' imported somefunc from 'a' before we >> monkey patched the module's attribute. Without a "from import", this >> would work: >> >> b.py: >> >> import a >> >> def someotherfunc(): >> return a.somefunc() + 'there' >> >> What I would normally resort to is patch somefunc's func_code, so that >> even code that used a "from import" before I could patch will use the >> patched version. > > I would probably patch b's somefunc. Note that setattr() will by > default raise an exception if "b.somefunc" does not exist. Good point. But this is where my example above actually differs from where I was trying to do this; namely in a fixture. So ideally all code paths that called somefunc would get the dummy. (I'm using this fixture in a functional test.) > As to patching func_code: good idea, i had forgotten about assigning func_code. > In earlier Python2.X versions assigning to func_code didn't work but i just > checked that it does on py27 and py33. So it's definitely worthwhile to > pursue. In your example you could do: > > monkeypatch.setattr('a.somefunc.func_code', (lambda: 'eh?').func_code) > > and it works (needs to use __code__ on py3). Nice. I hadn't thought about using setattr in this way! It's still a little unfortunate that monkeypatch.setattr works fine for very similar cases (e.g. class methods); just not for this one. At the same time, it's clearly following Python's own semantics and works just like setting an attribute. > I guess we could think about a > > monkeypatch.setcode("a.somefunc", lambda: 'eh?') > > helper. You could either pass in a function or a code object. > > We could even think about allowing non-string targets: > > monkeypatch.setcode(a.somefunc, lambda: 'eh?') > > Or would "monkeypatch.setfunc" be a better name? Hmm, I'm not sure how obscure my case is. And I guess even with a setcode in the API, people would still need to learn the hard way that the case of patching a module's attributes is special. I would certainly use setcode if it existed though. :-) Daniel From nicoddemus at gmail.com Thu Oct 24 01:51:41 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Wed, 23 Oct 2013 21:51:41 -0200 Subject: [pytest-dev] plugin status page In-Reply-To: References: <20131012131236.GK14010@merlinux.eu> <20131014203230.GX14010@merlinux.eu> <20131020055219.GB19953@merlinux.eu> Message-ID: Hi Holger, all, On Sun, Oct 20, 2013 at 11:06 AM, Bruno Oliveira wrote: > > On Sun, Oct 20, 2013 at 3:52 AM, holger krekel wrote: > >> >> What about generating a "plugin details " page for each plugin so we don't >> have to think too hard about what to put in the index page? >> > > That's a possibility, for sure. > > >> What is the advantage over just invoking "devpi test PLUGIN-NAME" from the >> script to automate the download/unpack/test/post-results steps? >> > > I didn't use devpi at that point because of my limited knowledge of devpi > at the moment, but that is certainly the next step. > > I will work on that next, and also work on generating a default tox.ini > file for plugins that don't have one as you suggested. > I have updated the script to create a tox.ini file that just uses "py.test --help" as a test command for those packages that don't have one. Now the script tests all pytest plugins in pypi, please take a look at the summary at the bottom of the page: https://www.travis-ci.org/nicoddemus/pytest-plugs/jobs/12959702 I guess it would be useful to identify which plugins are using just "py.test --help" as smoke test and which ones have real tests, since the former doesn't guarantee that the plugin actually works on that python/pytest combination. I looked into devpi to simplify the download/unpack/test process and stumbled in a minor roadblock: is there anyway to provide a tox.ini file for those plugins that don't have one? This is the only issue I see right now with using devpi for this particular application. > | > I guess we can implement in devpi a concept similar to travis build > status > >> > imagens, which we could then use in the plugins_index page. >> > >> > Against python 2.6 the script is failing, I have yet to investigate. >> > So, what do you think of the idea of using devpi.net to host test results and add an endpoint to the API to provide test status images? Or should I create another app to do this? Cheers, Bruno. > I found the reason for that (http vs https) and now the job runs against > Python 2.6, 2.7, 3.2 and 3.2. > > Thanks! Got back but caught a flue ... >> > > It happens, hope you get better soon. :) > > Cheers, > Bruno. > > > On Sun, Oct 20, 2013 at 3:52 AM, holger krekel wrote: > >> On Mon, Oct 14, 2013 at 23:52 -0300, Bruno Oliveira wrote: >> > > > > >> > > > > - i'd collapse NAME and VERSION columns to save space, i.e. >> > > > > "pytest-bdd-0.6.1". >> > > > > >> > > > > - what about adding download numbers? >> > > > > >> > > > > - as to code organisation: you can leave it as is for now or >> (maybe >> > > > > better) put all code and generated files into a dedicated >> directory >> > > > >> > > >> > >> > Done, opened a PR for discussion and reviewing. :) >> >> only got to it now, looks good so far! >> >> > > > > - we should try to collect repository locations. maybe parsing >> > > > > for github/bitbucket urls would yield most of them >> automatically? >> > > > > Maybe we need to add some manually. >> > > > > >> > > > >> > > > May I ask why we would need the repository locations? I mean, to >> work on >> > > > the compatibility feature we can work directly with packages in >> pypi or >> > > > devpi... >> > > >> > > purely for the person reading the page and wanting to look at the code >> > > or do a PR. >> > > >> > >> > Oh I see! Well, parsing the urls seems problematic, because we also >> have to >> > take in account the username ("hpk42/pytest" for example). I looked at >> the >> > GitHub and BitBucket's rest APIs to see if we could use them for >> searching, >> > but it seems only GitHub at the moment has a search API (and in beta >> stage >> > at that). >> > >> > I think we will have to maintain the repository list manually. Package >> > authors can help as well by simply opening PRs adding repositories for >> > their plugins I guess. >> >> What about generating a "plugin details " page for each plugin so we don't >> have to think too hard about what to put in the index page? >> >> > > A very interesting bit will be the "2.4.2 compat" and "dev" compat >> > > > >> > > >> > >> > Some late night quick hacking and I have come up with this proof of >> concept: >> > >> > https://www.travis-ci.org/nicoddemus/pytest-plugs >> > >> > As it is right now it serves only as a prove of concept, of course. The >> > idea is using travis to handle running things for us, driving it using a >> > script that downloads and run tests using tox. We can then collect json >> > information from that and POST those results back to devpi. Also we can >> use >> > its build matrix to test against different pytest versions, for example. >> >> Hum, interesting. Is this mainly to make use of travis-ci's build hosts? >> What is the advantage over just invoking "devpi test PLUGIN-NAME" from the >> script to automate the download/unpack/test/post-results steps? >> >> > I guess we can implement in devpi a concept similar to travis build >> status >> > imagens, which we could then use in the plugins_index page. >> > >> > Against python 2.6 the script is failing, I have yet to investigate. >> > >> > great, i am travelling to Pycon DE tomorrow morning but should be online >> > > from time to time. >> > > >> > >> > Nice tripe and good PyCon! >> >> Thanks! Got back but caught a flue ... >> >> cheers, >> holger >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Thu Oct 24 08:52:55 2013 From: holger at merlinux.eu (holger krekel) Date: Thu, 24 Oct 2013 06:52:55 +0000 Subject: [pytest-dev] plugin status page In-Reply-To: References: <20131012131236.GK14010@merlinux.eu> <20131014203230.GX14010@merlinux.eu> <20131020055219.GB19953@merlinux.eu> Message-ID: <20131024065255.GU3973@merlinux.eu> Hi Bruno, On Wed, Oct 23, 2013 at 21:51 -0200, Bruno Oliveira wrote: > Hi Holger, all, > > On Sun, Oct 20, 2013 at 11:06 AM, Bruno Oliveira wrote: > > > > > On Sun, Oct 20, 2013 at 3:52 AM, holger krekel wrote: > > > >> > >> What about generating a "plugin details " page for each plugin so we don't > >> have to think too hard about what to put in the index page? > >> > > > > That's a possibility, for sure. > > > > > >> What is the advantage over just invoking "devpi test PLUGIN-NAME" from the > >> script to automate the download/unpack/test/post-results steps? > >> > > > > I didn't use devpi at that point because of my limited knowledge of devpi > > at the moment, but that is certainly the next step. > > > > I will work on that next, and also work on generating a default tox.ini > > file for plugins that don't have one as you suggested. > > > > I have updated the script to create a tox.ini file that just uses "py.test > --help" as a test command for those packages that don't have one. Now the > script tests all pytest plugins in pypi, please take a look at the summary > at the bottom of the page: > > https://www.travis-ci.org/nicoddemus/pytest-plugs/jobs/12959702 looks good! I guess having a py27 and py33 row would be great to also determine py3 compatibility/installability. > I guess it would be useful to identify which plugins are using just > "py.test --help" as smoke test and which ones have real tests, since the > former doesn't guarantee that the plugin actually works on that > python/pytest combination. > > I looked into devpi to simplify the download/unpack/test process and > stumbled in a minor roadblock: is there anyway to provide a tox.ini file > for those plugins that don't have one? This is the only issue I see right > now with using devpi for this particular application. Now, but i am going to add a "devpi test -c path-to-ini" option today and plan to release devpi-1.2 real soon now. I'll get back here when it's implemented. > > | > I guess we can implement in devpi a concept similar to travis build > > status > > > >> > imagens, which we could then use in the plugins_index page. > >> > > >> > Against python 2.6 the script is failing, I have yet to investigate. > >> > > > So, what do you think of the idea of using devpi.net to host test results > and add an endpoint to the API to provide test status images? Or should I > create another app to do this? What do you mean with "test status images"? In general i am open to extending devpi-server to provide more test-status info -- just didn't have a chance to think about it in detail yet. I guess we could take the plugin testing bits as a starting point. best, holger > Cheers, > Bruno. > > > > I found the reason for that (http vs https) and now the job runs against > > Python 2.6, 2.7, 3.2 and 3.2. > > > > Thanks! Got back but caught a flue ... > >> > > > > It happens, hope you get better soon. :) > > > > Cheers, > > Bruno. > > > > > > On Sun, Oct 20, 2013 at 3:52 AM, holger krekel wrote: > > > >> On Mon, Oct 14, 2013 at 23:52 -0300, Bruno Oliveira wrote: > >> > > > > > >> > > > > - i'd collapse NAME and VERSION columns to save space, i.e. > >> > > > > "pytest-bdd-0.6.1". > >> > > > > > >> > > > > - what about adding download numbers? > >> > > > > > >> > > > > - as to code organisation: you can leave it as is for now or > >> (maybe > >> > > > > better) put all code and generated files into a dedicated > >> directory > >> > > > > >> > > > >> > > >> > Done, opened a PR for discussion and reviewing. :) > >> > >> only got to it now, looks good so far! > >> > >> > > > > - we should try to collect repository locations. maybe parsing > >> > > > > for github/bitbucket urls would yield most of them > >> automatically? > >> > > > > Maybe we need to add some manually. > >> > > > > > >> > > > > >> > > > May I ask why we would need the repository locations? I mean, to > >> work on > >> > > > the compatibility feature we can work directly with packages in > >> pypi or > >> > > > devpi... > >> > > > >> > > purely for the person reading the page and wanting to look at the code > >> > > or do a PR. > >> > > > >> > > >> > Oh I see! Well, parsing the urls seems problematic, because we also > >> have to > >> > take in account the username ("hpk42/pytest" for example). I looked at > >> the > >> > GitHub and BitBucket's rest APIs to see if we could use them for > >> searching, > >> > but it seems only GitHub at the moment has a search API (and in beta > >> stage > >> > at that). > >> > > >> > I think we will have to maintain the repository list manually. Package > >> > authors can help as well by simply opening PRs adding repositories for > >> > their plugins I guess. > >> > >> What about generating a "plugin details " page for each plugin so we don't > >> have to think too hard about what to put in the index page? > >> > >> > > A very interesting bit will be the "2.4.2 compat" and "dev" compat > >> > > > > >> > > > >> > > >> > Some late night quick hacking and I have come up with this proof of > >> concept: > >> > > >> > https://www.travis-ci.org/nicoddemus/pytest-plugs > >> > > >> > As it is right now it serves only as a prove of concept, of course. The > >> > idea is using travis to handle running things for us, driving it using a > >> > script that downloads and run tests using tox. We can then collect json > >> > information from that and POST those results back to devpi. Also we can > >> use > >> > its build matrix to test against different pytest versions, for example. > >> > >> Hum, interesting. Is this mainly to make use of travis-ci's build hosts? > >> What is the advantage over just invoking "devpi test PLUGIN-NAME" from the > >> script to automate the download/unpack/test/post-results steps? > >> > >> > I guess we can implement in devpi a concept similar to travis build > >> status > >> > imagens, which we could then use in the plugins_index page. > >> > > >> > Against python 2.6 the script is failing, I have yet to investigate. > >> > > >> > great, i am travelling to Pycon DE tomorrow morning but should be online > >> > > from time to time. > >> > > > >> > > >> > Nice tripe and good PyCon! > >> > >> Thanks! Got back but caught a flue ... > >> > >> cheers, > >> holger > >> > > > > From nicoddemus at gmail.com Thu Oct 24 09:08:08 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Thu, 24 Oct 2013 05:08:08 -0200 Subject: [pytest-dev] plugin status page In-Reply-To: <20131024065255.GU3973@merlinux.eu> References: <20131012131236.GK14010@merlinux.eu> <20131014203230.GX14010@merlinux.eu> <20131020055219.GB19953@merlinux.eu> <20131024065255.GU3973@merlinux.eu> Message-ID: Hi Holger, On Thu, Oct 24, 2013 at 4:52 AM, holger krekel wrote: > Hi Bruno, > > On Wed, Oct 23, 2013 at 21:51 -0200, Bruno Oliveira wrote: > > Hi Holger, all, > > > > > > I have updated the script to create a tox.ini file that just uses > "py.test > > --help" as a test command for those packages that don't have one. Now the > > script tests all pytest plugins in pypi, please take a look at the > summary > > at the bottom of the page: > > > > https://www.travis-ci.org/nicoddemus/pytest-plugs/jobs/12959702 > > looks good! I guess having a py27 and py33 row would be great > to also determine py3 compatibility/installability. > Those are already in place as different build jobs, please take a look: https://www.travis-ci.org/nicoddemus/pytest-plugs Currently we have py27 and py33 against pytest 2.3.5 and 2.4.2. Adding more python/pytest versions is easy, we just have to add them to the .travis file. :) https://github.com/nicoddemus/pytest-plugs/blob/master/.travis.yml > > > I guess it would be useful to identify which plugins are using just > > "py.test --help" as smoke test and which ones have real tests, since the > > former doesn't guarantee that the plugin actually works on that > > python/pytest combination. > > > > I looked into devpi to simplify the download/unpack/test process and > > stumbled in a minor roadblock: is there anyway to provide a tox.ini file > > for those plugins that don't have one? This is the only issue I see right > > now with using devpi for this particular application. > > Now, but i am going to add a "devpi test -c path-to-ini" option today > and plan to release devpi-1.2 real soon now. I'll get back here when it's > implemented. > Perfect, that would greatly simplify things. Let me know if there is anything I can do to help. > So, what do you think of the idea of using devpi.net to host test results > > and add an endpoint to the API to provide test status images? Or should I > > create another app to do this? > > What do you mean with "test status images"? In general i am open to extending devpi-server to provide more test-status > info -- just didn't have a chance to think about it in detail yet. > I guess we could take the plugin testing bits as a starting point. > I mean like those badge images served by travis that give the current build status, for example: https://secure.travis-ci.org/hpk42/pytest.png I think we can follow the same approach, generating a badge that gives the full compatibility status, like a single "py 2.6 2.7 | pytest 2.3.5 | compatible" image, or even multiple images. But I guess this is too specific to pytest... Cheers, Bruno. > > best, > holger > > > Cheers, > > Bruno. > > > > > > > I found the reason for that (http vs https) and now the job runs > against > > > Python 2.6, 2.7, 3.2 and 3.2. > > > > > > Thanks! Got back but caught a flue ... > > >> > > > > > > It happens, hope you get better soon. :) > > > > > > Cheers, > > > Bruno. > > > > > > > > > On Sun, Oct 20, 2013 at 3:52 AM, holger krekel > wrote: > > > > > >> On Mon, Oct 14, 2013 at 23:52 -0300, Bruno Oliveira wrote: > > >> > > > > > > >> > > > > - i'd collapse NAME and VERSION columns to save space, i.e. > > >> > > > > "pytest-bdd-0.6.1". > > >> > > > > > > >> > > > > - what about adding download numbers? > > >> > > > > > > >> > > > > - as to code organisation: you can leave it as is for now or > > >> (maybe > > >> > > > > better) put all code and generated files into a dedicated > > >> directory > > >> > > > > > >> > > > > >> > > > >> > Done, opened a PR for discussion and reviewing. :) > > >> > > >> only got to it now, looks good so far! > > >> > > >> > > > > - we should try to collect repository locations. maybe parsing > > >> > > > > for github/bitbucket urls would yield most of them > > >> automatically? > > >> > > > > Maybe we need to add some manually. > > >> > > > > > > >> > > > > > >> > > > May I ask why we would need the repository locations? I mean, to > > >> work on > > >> > > > the compatibility feature we can work directly with packages in > > >> pypi or > > >> > > > devpi... > > >> > > > > >> > > purely for the person reading the page and wanting to look at the > code > > >> > > or do a PR. > > >> > > > > >> > > > >> > Oh I see! Well, parsing the urls seems problematic, because we also > > >> have to > > >> > take in account the username ("hpk42/pytest" for example). I looked > at > > >> the > > >> > GitHub and BitBucket's rest APIs to see if we could use them for > > >> searching, > > >> > but it seems only GitHub at the moment has a search API (and in beta > > >> stage > > >> > at that). > > >> > > > >> > I think we will have to maintain the repository list manually. > Package > > >> > authors can help as well by simply opening PRs adding repositories > for > > >> > their plugins I guess. > > >> > > >> What about generating a "plugin details " page for each plugin so we > don't > > >> have to think too hard about what to put in the index page? > > >> > > >> > > A very interesting bit will be the "2.4.2 compat" and "dev" compat > > >> > > > > > >> > > > > >> > > > >> > Some late night quick hacking and I have come up with this proof of > > >> concept: > > >> > > > >> > https://www.travis-ci.org/nicoddemus/pytest-plugs > > >> > > > >> > As it is right now it serves only as a prove of concept, of course. > The > > >> > idea is using travis to handle running things for us, driving it > using a > > >> > script that downloads and run tests using tox. We can then collect > json > > >> > information from that and POST those results back to devpi. Also we > can > > >> use > > >> > its build matrix to test against different pytest versions, for > example. > > >> > > >> Hum, interesting. Is this mainly to make use of travis-ci's build > hosts? > > >> What is the advantage over just invoking "devpi test PLUGIN-NAME" > from the > > >> script to automate the download/unpack/test/post-results steps? > > >> > > >> > I guess we can implement in devpi a concept similar to travis build > > >> status > > >> > imagens, which we could then use in the plugins_index page. > > >> > > > >> > Against python 2.6 the script is failing, I have yet to investigate. > > >> > > > >> > great, i am travelling to Pycon DE tomorrow morning but should be > online > > >> > > from time to time. > > >> > > > > >> > > > >> > Nice tripe and good PyCon! > > >> > > >> Thanks! Got back but caught a flue ... > > >> > > >> cheers, > > >> holger > > >> > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Thu Oct 24 15:58:27 2013 From: holger at merlinux.eu (holger krekel) Date: Thu, 24 Oct 2013 13:58:27 +0000 Subject: [pytest-dev] plugin status page In-Reply-To: References: <20131012131236.GK14010@merlinux.eu> <20131014203230.GX14010@merlinux.eu> <20131020055219.GB19953@merlinux.eu> <20131024065255.GU3973@merlinux.eu> Message-ID: <20131024135827.GW3973@merlinux.eu> On Thu, Oct 24, 2013 at 05:08 -0200, Bruno Oliveira wrote: > Hi Holger, > > On Thu, Oct 24, 2013 at 4:52 AM, holger krekel wrote: > > > Hi Bruno, > > > > On Wed, Oct 23, 2013 at 21:51 -0200, Bruno Oliveira wrote: > > > Hi Holger, all, > > > > > > > > > I have updated the script to create a tox.ini file that just uses > > "py.test > > > --help" as a test command for those packages that don't have one. Now the > > > script tests all pytest plugins in pypi, please take a look at the > > summary > > > at the bottom of the page: > > > > > > https://www.travis-ci.org/nicoddemus/pytest-plugs/jobs/12959702 > > > > looks good! I guess having a py27 and py33 row would be great > > to also determine py3 compatibility/installability. > > > > Those are already in place as different build jobs, please take a look: > > https://www.travis-ci.org/nicoddemus/pytest-plugs Ah, see it now. > Currently we have py27 and py33 against pytest 2.3.5 and 2.4.2. Adding more > python/pytest versions is easy, we just have to add them to the .travis > file. :) > > https://github.com/nicoddemus/pytest-plugs/blob/master/.travis.yml OK. I think you can leave "--use-mirrors" away, btw. > > > > > I guess it would be useful to identify which plugins are using just > > > "py.test --help" as smoke test and which ones have real tests, since the > > > former doesn't guarantee that the plugin actually works on that > > > python/pytest combination. > > > > > > I looked into devpi to simplify the download/unpack/test process and > > > stumbled in a minor roadblock: is there anyway to provide a tox.ini file > > > for those plugins that don't have one? This is the only issue I see right > > > now with using devpi for this particular application. > > > > Now, but i am going to add a "devpi test -c path-to-ini" option today > > and plan to release devpi-1.2 real soon now. I'll get back here when it's > > implemented. > > > > Perfect, that would greatly simplify things. Let me know if there is > anything I can do to help. I just implemented it: https://bitbucket.org/hpk42/devpi/commits/3a8ff0503599358deeefedc581e88c03c4b94ae3 However, it's maybe not exactly what we need. "devpi test PKG" works by looking up the newest PKG release on the index, downloading and unpacking it and then running tox. Passing "devpi test -c tox.ini" will always prefer the external tox.ini, though. We could maybe add a second option "--fallback-ini" option that only is used if there is no tox.ini in the downloaded package. Note that to play with the devpi-{client,server} current trunk you can do: pip install --pre -i https://devpi.net/hpk/dev/+simple/ -U devpi and then run your own devpi-server similar to: http://doc.devpi.net/latest/quickstart-releaseprocess.html And especially try "devpi list PKG" after "devpi test PKG" to see test results. > > So, what do you think of the idea of using devpi.net to host test results > > > and add an endpoint to the API to provide test status images? Or should I > > > create another app to do this? > > > > What do you mean with "test status images"? > > In general i am open to extending devpi-server to provide more test-status > > info -- just didn't have a chance to think about it in detail yet. > > I guess we could take the plugin testing bits as a starting point. > > > > I mean like those badge images served by travis that give the current build > status, for example: > > https://secure.travis-ci.org/hpk42/pytest.png > > I think we can follow the same approach, generating a badge that gives the > full compatibility status, like a single "py 2.6 2.7 | pytest 2.3.5 | > compatible" image, or even multiple images. > > But I guess this is too specific to pytest... Hum, some badge makes sense. Not sure which, though :) holger > > Cheers, > Bruno. > > > > > > best, > > holger > > > > > Cheers, > > > Bruno. > > > > > > > > > > I found the reason for that (http vs https) and now the job runs > > against > > > > Python 2.6, 2.7, 3.2 and 3.2. > > > > > > > > Thanks! Got back but caught a flue ... > > > >> > > > > > > > > It happens, hope you get better soon. :) > > > > > > > > Cheers, > > > > Bruno. > > > > > > > > > > > > On Sun, Oct 20, 2013 at 3:52 AM, holger krekel > > wrote: > > > > > > > >> On Mon, Oct 14, 2013 at 23:52 -0300, Bruno Oliveira wrote: > > > >> > > > > > > > >> > > > > - i'd collapse NAME and VERSION columns to save space, i.e. > > > >> > > > > "pytest-bdd-0.6.1". > > > >> > > > > > > > >> > > > > - what about adding download numbers? > > > >> > > > > > > > >> > > > > - as to code organisation: you can leave it as is for now or > > > >> (maybe > > > >> > > > > better) put all code and generated files into a dedicated > > > >> directory > > > >> > > > > > > >> > > > > > >> > > > > >> > Done, opened a PR for discussion and reviewing. :) > > > >> > > > >> only got to it now, looks good so far! > > > >> > > > >> > > > > - we should try to collect repository locations. maybe parsing > > > >> > > > > for github/bitbucket urls would yield most of them > > > >> automatically? > > > >> > > > > Maybe we need to add some manually. > > > >> > > > > > > > >> > > > > > > >> > > > May I ask why we would need the repository locations? I mean, to > > > >> work on > > > >> > > > the compatibility feature we can work directly with packages in > > > >> pypi or > > > >> > > > devpi... > > > >> > > > > > >> > > purely for the person reading the page and wanting to look at the > > code > > > >> > > or do a PR. > > > >> > > > > > >> > > > > >> > Oh I see! Well, parsing the urls seems problematic, because we also > > > >> have to > > > >> > take in account the username ("hpk42/pytest" for example). I looked > > at > > > >> the > > > >> > GitHub and BitBucket's rest APIs to see if we could use them for > > > >> searching, > > > >> > but it seems only GitHub at the moment has a search API (and in beta > > > >> stage > > > >> > at that). > > > >> > > > > >> > I think we will have to maintain the repository list manually. > > Package > > > >> > authors can help as well by simply opening PRs adding repositories > > for > > > >> > their plugins I guess. > > > >> > > > >> What about generating a "plugin details " page for each plugin so we > > don't > > > >> have to think too hard about what to put in the index page? > > > >> > > > >> > > A very interesting bit will be the "2.4.2 compat" and "dev" compat > > > >> > > > > > > >> > > > > > >> > > > > >> > Some late night quick hacking and I have come up with this proof of > > > >> concept: > > > >> > > > > >> > https://www.travis-ci.org/nicoddemus/pytest-plugs > > > >> > > > > >> > As it is right now it serves only as a prove of concept, of course. > > The > > > >> > idea is using travis to handle running things for us, driving it > > using a > > > >> > script that downloads and run tests using tox. We can then collect > > json > > > >> > information from that and POST those results back to devpi. Also we > > can > > > >> use > > > >> > its build matrix to test against different pytest versions, for > > example. > > > >> > > > >> Hum, interesting. Is this mainly to make use of travis-ci's build > > hosts? > > > >> What is the advantage over just invoking "devpi test PLUGIN-NAME" > > from the > > > >> script to automate the download/unpack/test/post-results steps? > > > >> > > > >> > I guess we can implement in devpi a concept similar to travis build > > > >> status > > > >> > imagens, which we could then use in the plugins_index page. > > > >> > > > > >> > Against python 2.6 the script is failing, I have yet to investigate. > > > >> > > > > >> > great, i am travelling to Pycon DE tomorrow morning but should be > > online > > > >> > > from time to time. > > > >> > > > > > >> > > > > >> > Nice tripe and good PyCon! > > > >> > > > >> Thanks! Got back but caught a flue ... > > > >> > > > >> cheers, > > > >> holger > > > >> > > > > > > > > > > From nicoddemus at gmail.com Thu Oct 24 17:01:54 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Thu, 24 Oct 2013 13:01:54 -0200 Subject: [pytest-dev] plugin status page In-Reply-To: <20131024135827.GW3973@merlinux.eu> References: <20131012131236.GK14010@merlinux.eu> <20131014203230.GX14010@merlinux.eu> <20131020055219.GB19953@merlinux.eu> <20131024065255.GU3973@merlinux.eu> <20131024135827.GW3973@merlinux.eu> Message-ID: Hi Holger, On Thu, Oct 24, 2013 at 11:58 AM, holger krekel wrote: > OK. I think you can leave "--use-mirrors" away, btw. > I usually pass that parameter as is recommended by Travis' docs: "We highly recommend using --use-mirrors if you override dependency installation command to reduce the load on PyPI and possibility of installation failures." (from: http://about.travis-ci.org/docs/user/languages/python/#Travis-CI-uses-pip) Any reason to remove it that I'm not aware of? > Perfect, that would greatly simplify things. Let me know if there is > > anything I can do to help. > > I just implemented it: > > > https://bitbucket.org/hpk42/devpi/commits/3a8ff0503599358deeefedc581e88c03c4b94ae3 > > However, it's maybe not exactly what we need. "devpi test PKG" works by > looking up the newest PKG release on the index, downloading and unpacking > it and then running tox. Passing "devpi test -c tox.ini" will always > prefer the external tox.ini, though. We could maybe add a second option > "--fallback-ini" option that only is used if there is no tox.ini in the > downloaded package. > I guess we could execute that as a two step process, asking for devpi to download the package first to inspect if it has a tox.ini file, and change "devpi test" command line accordingly... but if you think implementing a "--fallback-ini" option is easy/useful for others, that's even better. Note that to play with the devpi-{client,server} current trunk you can do: > > pip install --pre -i https://devpi.net/hpk/dev/+simple/ -U devpi > > and then run your own devpi-server similar to: > > http://doc.devpi.net/latest/quickstart-releaseprocess.html > > And especially try "devpi list PKG" after "devpi test PKG" to see > test results. > Yes, have been playing around with it, very cool. :) > > I think we can follow the same approach, generating a badge that gives > the > > full compatibility status, like a single "py 2.6 2.7 | pytest 2.3.5 | > > compatible" image, or even multiple images. > > > > But I guess this is too specific to pytest... > > Hum, some badge makes sense. Not sure which, though :) > I was thinking that a "pytest compatible" badge is too specific for a generic tool like devpi, don't you think? Creating an app on Heroku to store test results from the travis jobs and serving appropriate badges seems simple enough... is this a direction you think we should take? Cheers, Bruno. holger > > > > > Cheers, > > Bruno. > > > > > > > > > > best, > > > holger > > > > > > > Cheers, > > > > Bruno. > > > > > > > > > > > > > I found the reason for that (http vs https) and now the job runs > > > against > > > > > Python 2.6, 2.7, 3.2 and 3.2. > > > > > > > > > > Thanks! Got back but caught a flue ... > > > > >> > > > > > > > > > > It happens, hope you get better soon. :) > > > > > > > > > > Cheers, > > > > > Bruno. > > > > > > > > > > > > > > > On Sun, Oct 20, 2013 at 3:52 AM, holger krekel > > > > wrote: > > > > > > > > > >> On Mon, Oct 14, 2013 at 23:52 -0300, Bruno Oliveira wrote: > > > > >> > > > > > > > > >> > > > > - i'd collapse NAME and VERSION columns to save space, > i.e. > > > > >> > > > > "pytest-bdd-0.6.1". > > > > >> > > > > > > > > >> > > > > - what about adding download numbers? > > > > >> > > > > > > > > >> > > > > - as to code organisation: you can leave it as is for now > or > > > > >> (maybe > > > > >> > > > > better) put all code and generated files into a > dedicated > > > > >> directory > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > Done, opened a PR for discussion and reviewing. :) > > > > >> > > > > >> only got to it now, looks good so far! > > > > >> > > > > >> > > > > - we should try to collect repository locations. maybe > parsing > > > > >> > > > > for github/bitbucket urls would yield most of them > > > > >> automatically? > > > > >> > > > > Maybe we need to add some manually. > > > > >> > > > > > > > > >> > > > > > > > >> > > > May I ask why we would need the repository locations? I > mean, to > > > > >> work on > > > > >> > > > the compatibility feature we can work directly with > packages in > > > > >> pypi or > > > > >> > > > devpi... > > > > >> > > > > > > >> > > purely for the person reading the page and wanting to look at > the > > > code > > > > >> > > or do a PR. > > > > >> > > > > > > >> > > > > > >> > Oh I see! Well, parsing the urls seems problematic, because we > also > > > > >> have to > > > > >> > take in account the username ("hpk42/pytest" for example). I > looked > > > at > > > > >> the > > > > >> > GitHub and BitBucket's rest APIs to see if we could use them for > > > > >> searching, > > > > >> > but it seems only GitHub at the moment has a search API (and in > beta > > > > >> stage > > > > >> > at that). > > > > >> > > > > > >> > I think we will have to maintain the repository list manually. > > > Package > > > > >> > authors can help as well by simply opening PRs adding > repositories > > > for > > > > >> > their plugins I guess. > > > > >> > > > > >> What about generating a "plugin details " page for each plugin so > we > > > don't > > > > >> have to think too hard about what to put in the index page? > > > > >> > > > > >> > > A very interesting bit will be the "2.4.2 compat" and "dev" > compat > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > Some late night quick hacking and I have come up with this > proof of > > > > >> concept: > > > > >> > > > > > >> > https://www.travis-ci.org/nicoddemus/pytest-plugs > > > > >> > > > > > >> > As it is right now it serves only as a prove of concept, of > course. > > > The > > > > >> > idea is using travis to handle running things for us, driving it > > > using a > > > > >> > script that downloads and run tests using tox. We can then > collect > > > json > > > > >> > information from that and POST those results back to devpi. > Also we > > > can > > > > >> use > > > > >> > its build matrix to test against different pytest versions, for > > > example. > > > > >> > > > > >> Hum, interesting. Is this mainly to make use of travis-ci's build > > > hosts? > > > > >> What is the advantage over just invoking "devpi test PLUGIN-NAME" > > > from the > > > > >> script to automate the download/unpack/test/post-results steps? > > > > >> > > > > >> > I guess we can implement in devpi a concept similar to travis > build > > > > >> status > > > > >> > imagens, which we could then use in the plugins_index page. > > > > >> > > > > > >> > Against python 2.6 the script is failing, I have yet to > investigate. > > > > >> > > > > > >> > great, i am travelling to Pycon DE tomorrow morning but should > be > > > online > > > > >> > > from time to time. > > > > >> > > > > > > >> > > > > > >> > Nice tripe and good PyCon! > > > > >> > > > > >> Thanks! Got back but caught a flue ... > > > > >> > > > > >> cheers, > > > > >> holger > > > > >> > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Thu Oct 24 22:39:35 2013 From: holger at merlinux.eu (holger krekel) Date: Thu, 24 Oct 2013 20:39:35 +0000 Subject: [pytest-dev] plugin status page In-Reply-To: References: <20131014203230.GX14010@merlinux.eu> <20131020055219.GB19953@merlinux.eu> <20131024065255.GU3973@merlinux.eu> <20131024135827.GW3973@merlinux.eu> Message-ID: <20131024203935.GY3973@merlinux.eu> Hi Bruno, On Thu, Oct 24, 2013 at 13:01 -0200, Bruno Oliveira wrote: > Hi Holger, > > On Thu, Oct 24, 2013 at 11:58 AM, holger krekel wrote: > > > OK. I think you can leave "--use-mirrors" away, btw. > > > > I usually pass that parameter as is recommended by Travis' docs: > > "We highly recommend using --use-mirrors if you override dependency > installation command to reduce the load on PyPI and possibility of > installation failures." > > (from: > http://about.travis-ci.org/docs/user/languages/python/#Travis-CI-uses-pip) > > Any reason to remove it that I'm not aware of? a) official mirrorrs are deprecated, see https://pypi.python.org/mirrors b) PyPI simple pages and release files are served through a CDN > > Perfect, that would greatly simplify things. Let me know if there is > > > anything I can do to help. > > > > I just implemented it: > > > > > > https://bitbucket.org/hpk42/devpi/commits/3a8ff0503599358deeefedc581e88c03c4b94ae3 > > > > However, it's maybe not exactly what we need. "devpi test PKG" works by > > looking up the newest PKG release on the index, downloading and unpacking > > it and then running tox. Passing "devpi test -c tox.ini" will always > > prefer the external tox.ini, though. We could maybe add a second option > > "--fallback-ini" option that only is used if there is no tox.ini in the > > downloaded package. > > > > I guess we could execute that as a two step process, asking for devpi to > download the package first to inspect if it has a tox.ini file, and change > "devpi test" command line accordingly... but if you think implementing a > "--fallback-ini" option is easy/useful for others, that's even better. I want to release devpi-1.2 any day now. Adding --fallback-ini as an experimental option is ok, i guess. Once we are through the testing plugin experience we can think about refining it. But others might want to use similar techniques (e.g. Pyramid plugins etc.). So i am going to add the fallback option probably tomorrow. > Note that to play with the devpi-{client,server} current trunk you can do: > > > > pip install --pre -i https://devpi.net/hpk/dev/+simple/ -U devpi > > > > and then run your own devpi-server similar to: > > > > http://doc.devpi.net/latest/quickstart-releaseprocess.html > > > > And especially try "devpi list PKG" after "devpi test PKG" to see > > test results. > > > > Yes, have been playing around with it, very cool. :) > > > > > I think we can follow the same approach, generating a badge that gives > > the > > > full compatibility status, like a single "py 2.6 2.7 | pytest 2.3.5 | > > > compatible" image, or even multiple images. > > > > > > But I guess this is too specific to pytest... > > > > Hum, some badge makes sense. Not sure which, though :) > > > > I was thinking that a "pytest compatible" badge is too specific for a > generic tool like devpi, don't you think? > > Creating an app on Heroku to store test results from the travis jobs and > serving appropriate badges seems simple enough... is this a direction you > think we should take? Might make sense. OTOH test results are already stored with devpi so at least it makes sense to try to use that information. Might need a bit of web API tweeks to make usage convenient. best, holger > > Cheers, > Bruno. > > holger > > > > > > > > Cheers, > > > Bruno. > > > > > > > > > > > > > > best, > > > > holger > > > > > > > > > Cheers, > > > > > Bruno. > > > > > > > > > > > > > > > > I found the reason for that (http vs https) and now the job runs > > > > against > > > > > > Python 2.6, 2.7, 3.2 and 3.2. > > > > > > > > > > > > Thanks! Got back but caught a flue ... > > > > > >> > > > > > > > > > > > > It happens, hope you get better soon. :) > > > > > > > > > > > > Cheers, > > > > > > Bruno. > > > > > > > > > > > > > > > > > > On Sun, Oct 20, 2013 at 3:52 AM, holger krekel > > > > > > wrote: > > > > > > > > > > > >> On Mon, Oct 14, 2013 at 23:52 -0300, Bruno Oliveira wrote: > > > > > >> > > > > > > > > > >> > > > > - i'd collapse NAME and VERSION columns to save space, > > i.e. > > > > > >> > > > > "pytest-bdd-0.6.1". > > > > > >> > > > > > > > > > >> > > > > - what about adding download numbers? > > > > > >> > > > > > > > > > >> > > > > - as to code organisation: you can leave it as is for now > > or > > > > > >> (maybe > > > > > >> > > > > better) put all code and generated files into a > > dedicated > > > > > >> directory > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > Done, opened a PR for discussion and reviewing. :) > > > > > >> > > > > > >> only got to it now, looks good so far! > > > > > >> > > > > > >> > > > > - we should try to collect repository locations. maybe > > parsing > > > > > >> > > > > for github/bitbucket urls would yield most of them > > > > > >> automatically? > > > > > >> > > > > Maybe we need to add some manually. > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > May I ask why we would need the repository locations? I > > mean, to > > > > > >> work on > > > > > >> > > > the compatibility feature we can work directly with > > packages in > > > > > >> pypi or > > > > > >> > > > devpi... > > > > > >> > > > > > > > >> > > purely for the person reading the page and wanting to look at > > the > > > > code > > > > > >> > > or do a PR. > > > > > >> > > > > > > > >> > > > > > > >> > Oh I see! Well, parsing the urls seems problematic, because we > > also > > > > > >> have to > > > > > >> > take in account the username ("hpk42/pytest" for example). I > > looked > > > > at > > > > > >> the > > > > > >> > GitHub and BitBucket's rest APIs to see if we could use them for > > > > > >> searching, > > > > > >> > but it seems only GitHub at the moment has a search API (and in > > beta > > > > > >> stage > > > > > >> > at that). > > > > > >> > > > > > > >> > I think we will have to maintain the repository list manually. > > > > Package > > > > > >> > authors can help as well by simply opening PRs adding > > repositories > > > > for > > > > > >> > their plugins I guess. > > > > > >> > > > > > >> What about generating a "plugin details " page for each plugin so > > we > > > > don't > > > > > >> have to think too hard about what to put in the index page? > > > > > >> > > > > > >> > > A very interesting bit will be the "2.4.2 compat" and "dev" > > compat > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > Some late night quick hacking and I have come up with this > > proof of > > > > > >> concept: > > > > > >> > > > > > > >> > https://www.travis-ci.org/nicoddemus/pytest-plugs > > > > > >> > > > > > > >> > As it is right now it serves only as a prove of concept, of > > course. > > > > The > > > > > >> > idea is using travis to handle running things for us, driving it > > > > using a > > > > > >> > script that downloads and run tests using tox. We can then > > collect > > > > json > > > > > >> > information from that and POST those results back to devpi. > > Also we > > > > can > > > > > >> use > > > > > >> > its build matrix to test against different pytest versions, for > > > > example. > > > > > >> > > > > > >> Hum, interesting. Is this mainly to make use of travis-ci's build > > > > hosts? > > > > > >> What is the advantage over just invoking "devpi test PLUGIN-NAME" > > > > from the > > > > > >> script to automate the download/unpack/test/post-results steps? > > > > > >> > > > > > >> > I guess we can implement in devpi a concept similar to travis > > build > > > > > >> status > > > > > >> > imagens, which we could then use in the plugins_index page. > > > > > >> > > > > > > >> > Against python 2.6 the script is failing, I have yet to > > investigate. > > > > > >> > > > > > > >> > great, i am travelling to Pycon DE tomorrow morning but should > > be > > > > online > > > > > >> > > from time to time. > > > > > >> > > > > > > > >> > > > > > > >> > Nice tripe and good PyCon! > > > > > >> > > > > > >> Thanks! Got back but caught a flue ... > > > > > >> > > > > > >> cheers, > > > > > >> holger > > > > > >> > > > > > > > > > > > > > > > > > > From flub at devork.be Mon Oct 28 22:44:10 2013 From: flub at devork.be (Floris Bruynooghe) Date: Mon, 28 Oct 2013 21:44:10 +0000 Subject: [pytest-dev] monkeypatch.setattr of functions and from imports In-Reply-To: <20131023183613.GS3973@merlinux.eu> References: <526807B7.4070900@gmail.com> <20131023183613.GS3973@merlinux.eu> Message-ID: On 23 October 2013 19:36, holger krekel wrote: > As to patching func_code: good idea, i had forgotten about assigning func_code. > In earlier Python2.X versions assigning to func_code didn't work but i just > checked that it does on py27 and py33. So it's definitely worthwhile to > pursue. In your example you could do: > > monkeypatch.setattr('a.somefunc.func_code', (lambda: 'eh?').func_code) > > and it works (needs to use __code__ on py3). I guess we could think about a > > monkeypatch.setcode("a.somefunc", lambda: 'eh?') > > helper. You could either pass in a function or a code object. > We could even think about allowing non-string targets: > > monkeypatch.setcode(a.somefunc, lambda: 'eh?') > > Or would "monkeypatch.setfunc" be a better name? Why does it have to be a new method? Can't setattr simply see if the object being patched as well as the object it is being patched with has a .func_code or .__code__ attribute and automatically patch the code if so? This would be additional to the current patching it does so that it's more obvious in a debugger for the current cases. It would have solved the OPs problem and I can't think of any real downside currently. Regards, Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From holger at merlinux.eu Tue Oct 29 06:39:02 2013 From: holger at merlinux.eu (holger krekel) Date: Tue, 29 Oct 2013 05:39:02 +0000 Subject: [pytest-dev] monkeypatch.setattr of functions and from imports In-Reply-To: References: <526807B7.4070900@gmail.com> <20131023183613.GS3973@merlinux.eu> Message-ID: <20131029053902.GO3973@merlinux.eu> On Mon, Oct 28, 2013 at 21:44 +0000, Floris Bruynooghe wrote: > On 23 October 2013 19:36, holger krekel wrote: > > As to patching func_code: good idea, i had forgotten about assigning func_code. > > In earlier Python2.X versions assigning to func_code didn't work but i just > > checked that it does on py27 and py33. So it's definitely worthwhile to > > pursue. In your example you could do: > > > > monkeypatch.setattr('a.somefunc.func_code', (lambda: 'eh?').func_code) > > > > and it works (needs to use __code__ on py3). I guess we could think about a > > > > monkeypatch.setcode("a.somefunc", lambda: 'eh?') > > > > helper. You could either pass in a function or a code object. > > We could even think about allowing non-string targets: > > > > monkeypatch.setcode(a.somefunc, lambda: 'eh?') > > > > Or would "monkeypatch.setfunc" be a better name? > > Why does it have to be a new method? Can't setattr simply see if the > object being patched as well as the object it is being patched with > has a .func_code or .__code__ attribute and automatically patch the > code if so? This would be additional to the current patching it does > so that it's more obvious in a debugger for the current cases. It > would have solved the OPs problem and I can't think of any real > downside currently. Wouldn't it violate what you expect from a "setattr" operation? Anyway, i played with setting __code__ the other day and there is a bad limitation:: def test_hello(monkeypatch): x = "/tmp" def myabs(somepath): return x > monkeypatch.setattr("os.path.abspath.__code__", myabs.__code__) E ValueError: abspath() requires a code object with 0 free vars, not 1 This means that setting __code__ with closure-using code does not work. This reduces the usefullness of setting code objects for practical purposes and speaks against introducing any support at this point. Setting non-closures might still be useful and the example above is other compatible to py26++ so it's probably good enough. best, holger From holger at merlinux.eu Wed Oct 30 11:42:13 2013 From: holger at merlinux.eu (holger krekel) Date: Wed, 30 Oct 2013 10:42:13 +0000 Subject: [pytest-dev] plugin status page In-Reply-To: <20131024203935.GY3973@merlinux.eu> References: <20131014203230.GX14010@merlinux.eu> <20131020055219.GB19953@merlinux.eu> <20131024065255.GU3973@merlinux.eu> <20131024135827.GW3973@merlinux.eu> <20131024203935.GY3973@merlinux.eu> Message-ID: <20131030104213.GR3973@merlinux.eu> On Thu, Oct 24, 2013 at 20:39 +0000, holger krekel wrote: > > I guess we could execute that as a two step process, asking for devpi to > > download the package first to inspect if it has a tox.ini file, and change > > "devpi test" command line accordingly... but if you think implementing a > > "--fallback-ini" option is easy/useful for others, that's even better. > > I want to release devpi-1.2 any day now. Adding --fallback-ini as > an experimental option is ok, i guess. Once we are through the testing > plugin experience we can think about refining it. But others might > want to use similar techniques (e.g. Pyramid plugins etc.). > So i am going to add the fallback option probably tomorrow. I just did devpi-*-1.2 releases. If you install devpi-client>=1.2 you get "devpi test --fallback-ini someini". Hope it helps. best, holger From nicoddemus at gmail.com Thu Oct 31 10:23:43 2013 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Thu, 31 Oct 2013 07:23:43 -0200 Subject: [pytest-dev] plugin status page In-Reply-To: <20131030104213.GR3973@merlinux.eu> References: <20131014203230.GX14010@merlinux.eu> <20131020055219.GB19953@merlinux.eu> <20131024065255.GU3973@merlinux.eu> <20131024135827.GW3973@merlinux.eu> <20131024203935.GY3973@merlinux.eu> <20131030104213.GR3973@merlinux.eu> Message-ID: It certainly helps! :) Cheers, Bruno On Wed, Oct 30, 2013 at 8:42 AM, holger krekel wrote: > On Thu, Oct 24, 2013 at 20:39 +0000, holger krekel wrote: > > > I guess we could execute that as a two step process, asking for devpi > to > > > download the package first to inspect if it has a tox.ini file, and > change > > > "devpi test" command line accordingly... but if you think implementing > a > > > "--fallback-ini" option is easy/useful for others, that's even better. > > > > I want to release devpi-1.2 any day now. Adding --fallback-ini as > > an experimental option is ok, i guess. Once we are through the testing > > plugin experience we can think about refining it. But others might > > want to use similar techniques (e.g. Pyramid plugins etc.). > > So i am going to add the fallback option probably tomorrow. > > I just did devpi-*-1.2 releases. If you install devpi-client>=1.2 > you get "devpi test --fallback-ini someini". Hope it helps. > > best, > holger > -------------- next part -------------- An HTML attachment was scrubbed... URL: