From me at jamescooke.info Wed Jan 6 12:36:28 2021 From: me at jamescooke.info (James Cooke) Date: Wed, 06 Jan 2021 17:36:28 +0000 Subject: [pytest-dev] Testing fixture teardown / finalisation Message-ID: Hi all, It's been a while since I've had to write a complicated and robust pytest fixture and I'm struggling with testing the teardown / finalisation code. Instead of boring you with the [my-work-project] requirements of cleaning up GCS after tests, I'll refer to the fixture in the docs https://docs.pytest.org/en/stable/fixture.html#fixture-finalization-executing-teardown-code : @pytest.fixture(scope="module") def smtp_connection(): smtp_connection = smtplib.SMTP("smtp.gmail.com", 587, timeout=5) yield smtp_connection # provide the fixture value print("teardown smtp") smtp_connection.close() My (very old) usual strategy for testing teardown of a fixture that uses `yield` would be: * Start test - Instantiate the fixture. - Manipulate the instance with `next()` to trigger the teardown. - Ensure that teardown was successful - in this case assert that the SMTP connection returned when the fixture was instantiated was closed successfully. Given that since v4.1 (https://github.com/pytest-dev/pytest/issues/4545), the arrangement step of "Instantiate the fixture" does not work, could someone point me at the recommended method of testing fixture teardown? I would like to ensure that teardown is resilient and can resolve multiple conditions that can happen in [my-work-project]'s test suite. I can see that there are multiple tests on fixtures in https://github.com/pytest-dev/pytest/blob/48c9a96a03261e7cfa5aad0367a9186d9032904a/testing/python/fixtures.py , are there any preferred / recommended / efficient methods in this file? Alternatively, is there a way to by-pass the fixture wrapping that happens that prevents it being callable? Should I be using `smtp_connection.__pytest_wrapped__.obj()` to instantiate? (this seems bad) Any pointers / suggestions would be great. Thanks, James From nicoddemus at gmail.com Wed Jan 6 13:26:08 2021 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Wed, 6 Jan 2021 15:26:08 -0300 Subject: [pytest-dev] Testing fixture teardown / finalisation In-Reply-To: References: Message-ID: Hi James, What I suggest you do in your case is to decouple your code from the fixture, so if you have something like this today: @pytest.fixture(scope="module") def my_work_reqs(): # some # complicated # setup # code yield x # some result # some # complicated # teardown # code You can write instead: @contextmanager def handle_work_reqs(): # some # complicated # setup # code try: yield x # some result finally: # some # complicated # teardown # code @pytest.fixture(scope="module") def my_work_reqs(): with handle_work_reqs() as x: yield x Now you can test handle_work_reqs normally, as it is not tied to the pytest framework. I like this approach in general, decoupling custom code from a framework is a great way to make it easier to test. HTH, Bruno. On Wed, Jan 6, 2021 at 2:37 PM James Cooke wrote: > Hi all, > > It's been a while since I've had to write a complicated and robust pytest > fixture and I'm struggling with testing the teardown / finalisation code. > > Instead of boring you with the [my-work-project] requirements of cleaning > up GCS after tests, I'll refer to the fixture in the docs > https://docs.pytest.org/en/stable/fixture.html#fixture-finalization-executing-teardown-code > : > > @pytest.fixture(scope="module") > def smtp_connection(): > smtp_connection = smtplib.SMTP("smtp.gmail.com", 587, timeout=5) > yield smtp_connection # provide the fixture value > print("teardown smtp") > smtp_connection.close() > > My (very old) usual strategy for testing teardown of a fixture that uses > `yield` would be: > > * Start test > > - Instantiate the fixture. > > - Manipulate the instance with `next()` to trigger the teardown. > > - Ensure that teardown was successful - in this case assert that the > SMTP connection returned when the fixture was instantiated was closed > successfully. > > Given that since v4.1 (https://github.com/pytest-dev/pytest/issues/4545), > the arrangement step of "Instantiate the fixture" does not work, could > someone point me at the recommended method of testing fixture teardown? > > I would like to ensure that teardown is resilient and can resolve multiple > conditions that can happen in [my-work-project]'s test suite. I can see > that there are multiple tests on fixtures in > https://github.com/pytest-dev/pytest/blob/48c9a96a03261e7cfa5aad0367a9186d9032904a/testing/python/fixtures.py > , are there any preferred / recommended / efficient methods in this file? > Alternatively, is there a way to by-pass the fixture wrapping that happens > that prevents it being callable? Should I be using > `smtp_connection.__pytest_wrapped__.obj()` to instantiate? (this seems bad) > > Any pointers / suggestions would be great. > > Thanks, > > James > > > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From variedthoughts at gmail.com Thu Jan 7 00:56:36 2021 From: variedthoughts at gmail.com (Brian Okken) Date: Wed, 6 Jan 2021 21:56:36 -0800 Subject: [pytest-dev] Testing fixture teardown / finalisation In-Reply-To: References: Message-ID: <3C6700C4-8FF8-4044-B67D-A05BF915335B@gmail.com> Bruno, This is brilliant in that it?s obvious now that I see it, but didn?t occur to me before. Super cool. - Brian > On Jan 6, 2021, at 10:26 AM, Bruno Oliveira wrote: > > ? > Hi James, > > What I suggest you do in your case is to decouple your code from the fixture, so if you have something like this today: > > @pytest.fixture(scope="module") > def my_work_reqs(): > # some > # complicated > # setup > # code > > yield x # some result > > # some > # complicated > # teardown > # code > You can write instead: > > > @contextmanager > def handle_work_reqs(): > # some > # complicated > # setup > # code > try: > yield x # some result > finally: > # some > # complicated > # teardown > # code > > @pytest.fixture(scope="module") > def my_work_reqs(): > with handle_work_reqs() as x: > yield x > Now you can test handle_work_reqs normally, as it is not tied to the pytest framework. > > I like this approach in general, decoupling custom code from a framework is a great way to make it easier to test. > > HTH, > Bruno. > > >> On Wed, Jan 6, 2021 at 2:37 PM James Cooke wrote: >> Hi all, >> >> It's been a while since I've had to write a complicated and robust pytest fixture and I'm struggling with testing the teardown / finalisation code. >> >> Instead of boring you with the [my-work-project] requirements of cleaning up GCS after tests, I'll refer to the fixture in the docs https://docs.pytest.org/en/stable/fixture.html#fixture-finalization-executing-teardown-code : >> >> @pytest.fixture(scope="module") >> def smtp_connection(): >> smtp_connection = smtplib.SMTP("smtp.gmail.com", 587, timeout=5) >> yield smtp_connection # provide the fixture value >> print("teardown smtp") >> smtp_connection.close() >> >> My (very old) usual strategy for testing teardown of a fixture that uses `yield` would be: >> >> * Start test >> >> - Instantiate the fixture. >> >> - Manipulate the instance with `next()` to trigger the teardown. >> >> - Ensure that teardown was successful - in this case assert that the SMTP connection returned when the fixture was instantiated was closed successfully. >> >> Given that since v4.1 (https://github.com/pytest-dev/pytest/issues/4545), the arrangement step of "Instantiate the fixture" does not work, could someone point me at the recommended method of testing fixture teardown? >> >> I would like to ensure that teardown is resilient and can resolve multiple conditions that can happen in [my-work-project]'s test suite. I can see that there are multiple tests on fixtures in https://github.com/pytest-dev/pytest/blob/48c9a96a03261e7cfa5aad0367a9186d9032904a/testing/python/fixtures.py , are there any preferred / recommended / efficient methods in this file? Alternatively, is there a way to by-pass the fixture wrapping that happens that prevents it being callable? Should I be using `smtp_connection.__pytest_wrapped__.obj()` to instantiate? (this seems bad) >> >> Any pointers / suggestions would be great. >> >> Thanks, >> >> James >> >> >> _______________________________________________ >> pytest-dev mailing list >> pytest-dev at python.org >> https://mail.python.org/mailman/listinfo/pytest-dev > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at jamescooke.info Thu Jan 7 09:57:45 2021 From: me at jamescooke.info (James Cooke) Date: Thu, 07 Jan 2021 14:57:45 +0000 Subject: [pytest-dev] Testing fixture teardown / finalisation In-Reply-To: <3C6700C4-8FF8-4044-B67D-A05BF915335B@gmail.com> References: <3C6700C4-8FF8-4044-B67D-A05BF915335B@gmail.com> Message-ID: <22fb21e9-fc26-40b4-a41d-3caccf917c75@www.fastmail.com> Hi Bruno, This is great! Just echoing what Brian said - simple and obvious, and I'm disappointed that I didn't see it myself. I've integrated your suggestion today and the teardown tests are now much simpler, so this definitely helped. Would it be worth PR-ing this suggestion into the docs? My guess is that it would help many devs. Am happy to do the copypasta if you think it's worth it. Cheers, James On Thu, 7 Jan 2021, at 5:56 AM, Brian Okken wrote: > Bruno, > > This is brilliant in that it?s obvious now that I see it, but didn?t occur to me before. > > Super cool. > > - Brian > >> On Jan 6, 2021, at 10:26 AM, Bruno Oliveira wrote: >> ? >> Hi James, >> What I suggest you do in your case is to decouple your code from the fixture, so if you have something like this today: >> ` @pytest.fixture(scope="module") def my_work_reqs(): # some # complicated # setup # code yield x # some result # some # complicated # teardown # code ` >> You can write instead: >> ` @contextmanager def handle_work_reqs(): # some # complicated # setup # code try: yield x # some result finally: # some # complicated # teardown # code @pytest.fixture(scope="module") def my_work_reqs(): with handle_work_reqs() as x: yield x ` >> Now you can test `handle_work_reqs` normally, as it is not tied to the pytest framework. >> I like this approach in general, decoupling custom code from a framework is a great way to make it easier to test. >> HTH, >> Bruno. >> >> >> On Wed, Jan 6, 2021 at 2:37 PM James Cooke wrote: >>> Hi all, >>> >>> It's been a while since I've had to write a complicated and robust pytest fixture and I'm struggling with testing the teardown / finalisation code. >>> >>> Instead of boring you with the [my-work-project] requirements of cleaning up GCS after tests, I'll refer to the fixture in the docs https://docs.pytest.org/en/stable/fixture.html#fixture-finalization-executing-teardown-code : >>> >>> @pytest.fixture(scope="module") >>> def smtp_connection(): >>> smtp_connection = smtplib.SMTP("smtp.gmail.com", 587, timeout=5) >>> yield smtp_connection # provide the fixture value >>> print("teardown smtp") >>> smtp_connection.close() >>> >>> My (very old) usual strategy for testing teardown of a fixture that uses `yield` would be: >>> >>> * Start test >>> >>> - Instantiate the fixture. >>> >>> - Manipulate the instance with `next()` to trigger the teardown. >>> >>> - Ensure that teardown was successful - in this case assert that the SMTP connection returned when the fixture was instantiated was closed successfully. >>> >>> Given that since v4.1 (https://github.com/pytest-dev/pytest/issues/4545), the arrangement step of "Instantiate the fixture" does not work, could someone point me at the recommended method of testing fixture teardown? >>> >>> I would like to ensure that teardown is resilient and can resolve multiple conditions that can happen in [my-work-project]'s test suite. I can see that there are multiple tests on fixtures in https://github.com/pytest-dev/pytest/blob/48c9a96a03261e7cfa5aad0367a9186d9032904a/testing/python/fixtures.py , are there any preferred / recommended / efficient methods in this file? Alternatively, is there a way to by-pass the fixture wrapping that happens that prevents it being callable? Should I be using `smtp_connection.__pytest_wrapped__.obj()` to instantiate? (this seems bad) >>> >>> Any pointers / suggestions would be great. >>> >>> Thanks, >>> >>> James >>> >>> >>> _______________________________________________ >>> pytest-dev mailing list >>> pytest-dev at python.org >>> https://mail.python.org/mailman/listinfo/pytest-dev >> _______________________________________________ >> pytest-dev mailing list >> pytest-dev at python.org >> https://mail.python.org/mailman/listinfo/pytest-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicoddemus at gmail.com Thu Jan 7 10:35:15 2021 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Thu, 7 Jan 2021 12:35:15 -0300 Subject: [pytest-dev] Testing fixture teardown / finalisation In-Reply-To: <22fb21e9-fc26-40b4-a41d-3caccf917c75@www.fastmail.com> References: <3C6700C4-8FF8-4044-B67D-A05BF915335B@gmail.com> <22fb21e9-fc26-40b4-a41d-3caccf917c75@www.fastmail.com> Message-ID: Hi Brian, James, Glad you both found it useful! > Would it be worth PR-ing this suggestion into the docs? Sure, sounds great. While this is definitely useful for this use case, it is also a good generic testing advice, so definitely has a place in pytest's docs. Cheers, Bruno. On Thu, Jan 7, 2021 at 11:58 AM James Cooke wrote: > Hi Bruno, > > This is great! Just echoing what Brian said - simple and obvious, and I'm > disappointed that I didn't see it myself. I've integrated your suggestion > today and the teardown tests are now much simpler, so this definitely > helped. > > Would it be worth PR-ing this suggestion into the docs? My guess is that > it would help many devs. Am happy to do the copypasta if you think it's > worth it. > > Cheers, > > James > > > On Thu, 7 Jan 2021, at 5:56 AM, Brian Okken wrote: > > Bruno, > > This is brilliant in that it?s obvious now that I see it, but didn?t occur > to me before. > > Super cool. > > - Brian > > On Jan 6, 2021, at 10:26 AM, Bruno Oliveira wrote: > > ? > > Hi James, > > What I suggest you do in your case is to decouple your code from the > fixture, so if you have something like this today: > > @pytest.fixture(scope="module") > def my_work_reqs(): > # some > # complicated > # setup > # code > > yield x # some result > > # some > # complicated > # teardown > # code > > You can write instead: > > > @contextmanager > def handle_work_reqs(): > # some > # complicated > # setup > # code > try: > yield x # some result > finally: > # some > # complicated > # teardown > # code > > @pytest.fixture(scope="module") > def my_work_reqs(): > with handle_work_reqs() as x: > yield x > > Now you can test handle_work_reqs normally, as it is not tied to the > pytest framework. > > I like this approach in general, decoupling custom code from a framework > is a great way to make it easier to test. > > HTH, > Bruno. > > > > On Wed, Jan 6, 2021 at 2:37 PM James Cooke wrote: > > Hi all, > > It's been a while since I've had to write a complicated and robust pytest > fixture and I'm struggling with testing the teardown / finalisation code. > > Instead of boring you with the [my-work-project] requirements of cleaning > up GCS after tests, I'll refer to the fixture in the docs > https://docs.pytest.org/en/stable/fixture.html#fixture-finalization-executing-teardown-code > : > > @pytest.fixture(scope="module") > def smtp_connection(): > smtp_connection = smtplib.SMTP("smtp.gmail.com", 587, timeout=5) > yield smtp_connection # provide the fixture value > print("teardown smtp") > smtp_connection.close() > > My (very old) usual strategy for testing teardown of a fixture that uses > `yield` would be: > > * Start test > > - Instantiate the fixture. > > - Manipulate the instance with `next()` to trigger the teardown. > > - Ensure that teardown was successful - in this case assert that the > SMTP connection returned when the fixture was instantiated was closed > successfully. > > Given that since v4.1 (https://github.com/pytest-dev/pytest/issues/4545), > the arrangement step of "Instantiate the fixture" does not work, could > someone point me at the recommended method of testing fixture teardown? > > I would like to ensure that teardown is resilient and can resolve multiple > conditions that can happen in [my-work-project]'s test suite. I can see > that there are multiple tests on fixtures in > https://github.com/pytest-dev/pytest/blob/48c9a96a03261e7cfa5aad0367a9186d9032904a/testing/python/fixtures.py > , are there any preferred / recommended / efficient methods in this file? > Alternatively, is there a way to by-pass the fixture wrapping that happens > that prevents it being callable? Should I be using > `smtp_connection.__pytest_wrapped__.obj()` to instantiate? (this seems bad) > > Any pointers / suggestions would be great. > > Thanks, > > James > > > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at the-compiler.org Thu Jan 7 12:11:53 2021 From: me at the-compiler.org (Florian Bruhin) Date: Thu, 7 Jan 2021 18:11:53 +0100 Subject: [pytest-dev] Testing fixture teardown / finalisation In-Reply-To: References: Message-ID: <20210107171153.smv44wa3jzu6tqcg@aragog.localdomain> Hey, On Wed, Jan 06, 2021 at 03:26:08PM -0300, Bruno Oliveira wrote: > What I suggest you do in your case is to decouple your code from the > fixture, so if you have something like this today: > > [setup / teardown in a fixture] > > You can write instead: > > @contextmanager > def handle_work_reqs(): > # [setup] > try: > yield x # some result > finally: > # [teardown] > > @pytest.fixture(scope="module") > def my_work_reqs(): > with handle_work_reqs() as x: > yield x Perhaps it's just as obvious (at least in hindsight), but explicit is better as implicit: Another thing I often do is return some kind of custom class from the fixture, which tests can use - let's say we have: class JSTester: def __init__(self, qtbot): self.qtbot = qtbot # a fixture needed in the class def run(self, source): # ... # [some other methods tests can use] @pytest.fixture def js_tester(qtbot): return JSTester(qtbot) So that the tests can do things like: def test_simple_js(js_tester): js_tester.run('1 + 1') If there was some teardown to do here, I'd just add a "cleanup" method to that class, and call that in the fixture. If, for some reason, you have some more complex scenario and want to test it inside pytest rather than isolated from it, another possibility is the pytester fixture (called testdir in earlier releases): https://docs.pytest.org/en/latest/writing_plugins.html#testing-plugins https://docs.pytest.org/en/latest/reference.html#std-fixture-pytester Florian -- me at the-compiler.org (Mail/XMPP) | https://www.qutebrowser.org https://bruhin.software/ | https://github.com/sponsors/The-Compiler/ GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc I love long mails! | https://email.is-not-s.ms/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From flub at devork.be Thu Jan 7 15:00:44 2021 From: flub at devork.be (Floris Bruynooghe) Date: Thu, 07 Jan 2021 21:00:44 +0100 Subject: [pytest-dev] Testing fixture teardown / finalisation In-Reply-To: <20210107171153.smv44wa3jzu6tqcg@aragog.localdomain> References: <20210107171153.smv44wa3jzu6tqcg@aragog.localdomain> Message-ID: <8735zc5xg3.fsf@powell.devork.be> Hello, I can recommend both Bruno's and Florian's approaches as being good practice :) Cheers, Floris On Thu 07 Jan 2021 at 18:11 +0100, Florian Bruhin wrote: > Hey, > > On Wed, Jan 06, 2021 at 03:26:08PM -0300, Bruno Oliveira wrote: >> What I suggest you do in your case is to decouple your code from the >> fixture, so if you have something like this today: >> >> [setup / teardown in a fixture] >> >> You can write instead: >> >> @contextmanager >> def handle_work_reqs(): >> # [setup] >> try: >> yield x # some result >> finally: >> # [teardown] >> >> @pytest.fixture(scope="module") >> def my_work_reqs(): >> with handle_work_reqs() as x: >> yield x > > Perhaps it's just as obvious (at least in hindsight), but explicit is > better as implicit: > > Another thing I often do is return some kind of custom class from the > fixture, which tests can use - let's say we have: > > class JSTester: > > def __init__(self, qtbot): > self.qtbot = qtbot # a fixture needed in the class > > def run(self, source): > # ... > > # [some other methods tests can use] > > @pytest.fixture > def js_tester(qtbot): > return JSTester(qtbot) > > So that the tests can do things like: > > def test_simple_js(js_tester): > js_tester.run('1 + 1') > > If there was some teardown to do here, I'd just add a "cleanup" method > to that class, and call that in the fixture. > > If, for some reason, you have some more complex scenario and want to > test it inside pytest rather than isolated from it, another possibility > is the pytester fixture (called testdir in earlier releases): > > https://docs.pytest.org/en/latest/writing_plugins.html#testing-plugins > https://docs.pytest.org/en/latest/reference.html#std-fixture-pytester > > Florian From nicoddemus at gmail.com Mon Jan 25 09:54:50 2021 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Mon, 25 Jan 2021 11:54:50 -0300 Subject: [pytest-dev] pytest-6.2.2 Message-ID: pytest 6.2.2 has just been released to PyPI. This is a bug-fix release, being a drop-in replacement. To upgrade:: pip install --upgrade pytest The full changelog is available at https://docs.pytest.org/en/stable/changelog.html. Thanks to all of the contributors to this release: * Adam Johnson * Bruno Oliveira * Chris NeJame * Ran Benita Happy testing, The pytest Development Team -------------- next part -------------- An HTML attachment was scrubbed... URL: