From flub at devork.be Sun Dec 2 06:56:10 2018 From: flub at devork.be (Floris Bruynooghe) Date: Sun, 02 Dec 2018 12:56:10 +0100 Subject: [pytest-dev] enhancing the assertion rewriting experience with custom integration/workflows In-Reply-To: References: Message-ID: <878t189m79.fsf@powell.devork.be> Hi Ronny, On Thu 29 Nov 2018 at 15:57 +0100, Ronny Pfannschmidt wrote: > at a project at work which integrates pytest > we did build a custom workflow around invoking tests and pytest. > > This in turn triggers a lot of warnings from the assertion rewriter about > things that have been already imported. > > at first glance fixing this seems rather problematic, as the configuration > of the assertion rewriting is deeply bound to the pytest configuration and > it seems we cant do something simple and crude like letting a pth file fix > the issue. > > i believe opening up the hooking system could be best done by moving it to > a package with some sort of singleton control anyway - then pytest would be > just a consumer of that api enabling if not enabled and push in its own > "import roots". Having the assertion-rewriting in a separate package has been considered many times already and I can't remember anyone who was actually against this. I think it's just been waiting for someone to actually really need it and thus do it and have at least two consumers of the API. So do you have any more ideas about what it would look like? As a random start I think some of these things should be in it: - The AST-rewriter. Maybe a little more generalised than the pytest version, or at least allow for the API to grow in that direction. E.g customise the code with which the assert statement gets re-written. - The import hook. I imagine this would not be installed at all as this would be up to the user of the package, e.g. pytest. I'm not sure if this second part is what you had in mind, since it's still pytest which would install the import hook. But your problem description is a bit vague, so I'm not entirely sure what your aim is. Cheers, Floris From sivan at vitakka.co Mon Dec 3 03:15:19 2018 From: sivan at vitakka.co (=?UTF-8?Q?Sivan_Gr=C3=BCnberg?=) Date: Mon, 3 Dec 2018 10:15:19 +0200 Subject: [pytest-dev] Looking for additional maintainers for pytest plugins In-Reply-To: References: Message-ID: Hi Dave, I'm in to help as well, can probably handle any of the plugins. -Sivan On Fri, Nov 30, 2018 at 9:20 PM Dave Hunt wrote: > Due to a change in my role, I?m no longer going to be able to spend as > much time as I have with maintaining the plugins listed below. The > maintenance burden is very light, and releases are handled through Travis > CI. I?ve already had some excellent help from Jim Br?nnlund (BeyondEvil), > and have added him as a developer to a few of these. I?d appreciate any > additional help from the community to support these plugins, and if you?re > aware of somebody who would like to contribute please point them in my > direction. > > https://github.com/pytest-dev/pytest-base-url - A very small plugin, no > commits in over a year > https://github.com/pytest-dev/pytest-selenium - Jim is a developer, and > helps out a lot with this one, he's taken care of much of the recent > development and responding to issues > https://github.com/pytest-dev/pytest-html - This is probably the most > popular plugin I?ve been maintaining, and I think there?s a lot of > potential for enhancements (great ideas already in the tracker) > https://github.com/pytest-dev/pytest-metadata - Split off as a dependency > for pytest-html, relatively small plugin and minimal maintenance needs. Jim > is a developer for this plugin too > https://github.com/pytest-dev/pytest-variables - Another small plugin > with very few maintenance needs > https://github.com/pytest-dev/pytest-repeat - I don?t really maintain > this one currently, but feel that it?s worth mentioning because I was one > of the early contributors > > I?m happy to answer any questions, and keen to hear any suggestions for > handling any handover > > I?m sure these are in safe hands, and I will continue to use them myself, > and promote the use of them to others! > > Cheers, > Dave > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > -- Sivan Greenberg Co founder & CTO Vitakka Consulting -------------- next part -------------- An HTML attachment was scrubbed... URL: From sivan at vitakka.co Mon Dec 3 03:24:25 2018 From: sivan at vitakka.co (=?UTF-8?Q?Sivan_Gr=C3=BCnberg?=) Date: Mon, 3 Dec 2018 10:24:25 +0200 Subject: [pytest-dev] document outlining added values Message-ID: Hi All, A transcript from a conversation about considering porting django TestCase's to pytest: "Some counterpoints: - Tomorrow `some_other_test` will be the guideline and pytest will be passe - Why add a new dependency when you have something built in, _if it's good enough_ But all that is just conversation, the real issue i'm having is: Now we have 2 ways of doing something in the project consistency is my #1 goal or requirement in a project Makes the codebase so much easier to maintain and learn (edited) I'm trying to grok if pytest gives us such advantages that it's worth gradually porting all tests to use it for consistency, but while the syntax is nice you could achieve the same thing by reading those paramterized lists in a loop i think?" Last remark was in reference to @pytest.mark.parametrize. Do we have a document somewhere to answer such? like a comparison that outlines why it might be better to do so , or alternatively, to discuss the contexts where sticking to Django TestCase's is better? Is there a full feature list that I may point people at when they ask me "why pytest" ? Cheers, -Sivan -------------- next part -------------- An HTML attachment was scrubbed... URL: From craig.ds at gmail.com Mon Dec 3 19:54:50 2018 From: craig.ds at gmail.com (Craig de Stigter) Date: Tue, 4 Dec 2018 00:54:50 +0000 Subject: [pytest-dev] document outlining added values In-Reply-To: References: Message-ID: > gradually porting all tests Mostly-automated conversion is possible, so it doesn't have to be a gradual process. I wrote a script which converts xunit-style assertions into pytest ones: https://github.com/craigds/decrapify/blob/master/pytestify.py In the context of a large project port, my PR for OSGeo/gdal is under consideration for merge. Most of the work was automated. I would expect a conversion of django's test suite to pytest to be simpler than the GDAL one (django's test suite is in far better shape to start with). As a heavy user of django, I'd be interested in taking the lead on it, or at least contributing in an advisory way. I'm not aware of any discussion on the django lists about this yet though. Is your question purely academic or are you considering submitting something to Django to work on this? Cheers Craig de Stigter On Mon, 3 Dec 2018 at 21:24 Sivan Gr?nberg wrote: > Hi All, > > A transcript from a conversation about considering porting django > TestCase's to pytest: > > "Some counterpoints: > - Tomorrow `some_other_test` will be the guideline and pytest will be passe > - Why add a new dependency when you have something built in, _if it's good > enough_ > > But all that is just conversation, the real issue i'm having is: > Now we have 2 ways of doing something in the project > consistency is my #1 goal or requirement in a project > Makes the codebase so much easier to maintain and learn (edited) > I'm trying to grok if pytest gives us such advantages that it's worth > gradually porting all tests to use it for consistency, but while the syntax > is nice you could achieve the same thing by reading those paramterized > lists in a loop i think?" > > Last remark was in reference to @pytest.mark.parametrize. > > Do we have a document somewhere to answer such? like a comparison that > outlines why it might be better to do so , or alternatively, to discuss the > contexts where sticking to Django TestCase's is better? > > Is there a full feature list that I may point people at when they ask me > "why pytest" ? > > Cheers, > > -Sivan > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From craig.ds at gmail.com Mon Dec 3 19:56:24 2018 From: craig.ds at gmail.com (Craig de Stigter) Date: Tue, 4 Dec 2018 00:56:24 +0000 Subject: [pytest-dev] document outlining added values In-Reply-To: References:

Message-ID: Just after posting I re-read your post, and maybe I mis-read it the first time. Are you referring to converting a django-based project, rather than Django itself? On Tue, 4 Dec 2018 at 13:54 Craig de Stigter wrote: > > gradually porting all tests > > Mostly-automated conversion is possible, so it doesn't have to be a > gradual process. > > I wrote a script which converts xunit-style assertions into pytest ones: > https://github.com/craigds/decrapify/blob/master/pytestify.py > > In the context of a large project port, my PR for OSGeo/gdal > is under consideration for > merge. Most of the work was automated. > > I would expect a conversion of django's test suite to pytest to be simpler > than the GDAL one (django's test suite is in far better shape to start > with). As a heavy user of django, I'd be interested in taking the lead on > it, or at least contributing in an advisory way. I'm not aware of any > discussion on the django lists about this yet though. > > Is your question purely academic or are you considering submitting > something to Django to work on this? > > Cheers > Craig de Stigter > > On Mon, 3 Dec 2018 at 21:24 Sivan Gr?nberg wrote: > >> Hi All, >> >> A transcript from a conversation about considering porting django >> TestCase's to pytest: >> >> "Some counterpoints: >> - Tomorrow `some_other_test` will be the guideline and pytest will be >> passe >> - Why add a new dependency when you have something built in, _if it's >> good enough_ >> >> But all that is just conversation, the real issue i'm having is: >> Now we have 2 ways of doing something in the project >> consistency is my #1 goal or requirement in a project >> Makes the codebase so much easier to maintain and learn (edited) >> I'm trying to grok if pytest gives us such advantages that it's worth >> gradually porting all tests to use it for consistency, but while the syntax >> is nice you could achieve the same thing by reading those paramterized >> lists in a loop i think?" >> >> Last remark was in reference to @pytest.mark.parametrize. >> >> Do we have a document somewhere to answer such? like a comparison that >> outlines why it might be better to do so , or alternatively, to discuss the >> contexts where sticking to Django TestCase's is better? >> >> Is there a full feature list that I may point people at when they ask me >> "why pytest" ? >> >> Cheers, >> >> -Sivan >> _______________________________________________ >> pytest-dev mailing list >> pytest-dev at python.org >> https://mail.python.org/mailman/listinfo/pytest-dev >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sivan at vitakka.co Tue Dec 4 02:29:09 2018 From: sivan at vitakka.co (=?UTF-8?Q?Sivan_Gr=C3=BCnberg?=) Date: Tue, 4 Dec 2018 09:29:09 +0200 Subject: [pytest-dev] document outlining added values In-Reply-To: References:

Message-ID: Yes, converting a django based project. On Tue, 4 Dec 2018 at 2:56 Craig de Stigter wrote: > Just after posting I re-read your post, and maybe I mis-read it the first > time. Are you referring to converting a django-based project, rather than > Django itself? > > On Tue, 4 Dec 2018 at 13:54 Craig de Stigter wrote: > >> > gradually porting all tests >> >> Mostly-automated conversion is possible, so it doesn't have to be a >> gradual process. >> >> I wrote a script which converts xunit-style assertions into pytest ones: >> https://github.com/craigds/decrapify/blob/master/pytestify.py >> >> In the context of a large project port, my PR for OSGeo/gdal >> is under consideration for >> merge. Most of the work was automated. >> >> I would expect a conversion of django's test suite to pytest to be >> simpler than the GDAL one (django's test suite is in far better shape to >> start with). As a heavy user of django, I'd be interested in taking the >> lead on it, or at least contributing in an advisory way. I'm not aware of >> any discussion on the django lists about this yet though. >> >> Is your question purely academic or are you considering submitting >> something to Django to work on this? >> >> Cheers >> Craig de Stigter >> >> On Mon, 3 Dec 2018 at 21:24 Sivan Gr?nberg wrote: >> >>> Hi All, >>> >>> A transcript from a conversation about considering porting django >>> TestCase's to pytest: >>> >>> "Some counterpoints: >>> - Tomorrow `some_other_test` will be the guideline and pytest will be >>> passe >>> - Why add a new dependency when you have something built in, _if it's >>> good enough_ >>> >>> But all that is just conversation, the real issue i'm having is: >>> Now we have 2 ways of doing something in the project >>> consistency is my #1 goal or requirement in a project >>> Makes the codebase so much easier to maintain and learn (edited) >>> I'm trying to grok if pytest gives us such advantages that it's worth >>> gradually porting all tests to use it for consistency, but while the syntax >>> is nice you could achieve the same thing by reading those paramterized >>> lists in a loop i think?" >>> >>> Last remark was in reference to @pytest.mark.parametrize. >>> >>> Do we have a document somewhere to answer such? like a comparison that >>> outlines why it might be better to do so , or alternatively, to discuss the >>> contexts where sticking to Django TestCase's is better? >>> >>> Is there a full feature list that I may point people at when they ask me >>> "why pytest" ? >>> >>> Cheers, >>> >>> -Sivan >>> _______________________________________________ >>> pytest-dev mailing list >>> pytest-dev at python.org >>> https://mail.python.org/mailman/listinfo/pytest-dev >>> >> -- Sivan Greenberg Co founder & CTO Vitakka Consulting -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at the-compiler.org Tue Dec 4 07:12:16 2018 From: me at the-compiler.org (Florian Bruhin) Date: Tue, 4 Dec 2018 13:12:16 +0100 Subject: [pytest-dev] document outlining added values In-Reply-To: References:

Message-ID: <20181204121216.o3isvqmd35j3q2ex@hooch.localdomain> On Tue, Dec 04, 2018 at 12:54:50AM +0000, Craig de Stigter wrote: > > gradually porting all tests > > Mostly-automated conversion is possible, so it doesn't have to be a gradual > process. > > I wrote a script which converts xunit-style assertions into pytest ones: > https://github.com/craigds/decrapify/blob/master/pytestify.py Yet another one? ;-) https://github.com/pytest-dev/unittest2pytest https://github.com/dropbox/dbx-unittest2pytest Florian -- https://www.qutebrowser.org | me at the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc I love long mails! | https://email.is-not-s.ms/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From craig.ds at gmail.com Tue Dec 4 15:37:27 2018 From: craig.ds at gmail.com (Craig de Stigter) Date: Wed, 5 Dec 2018 09:37:27 +1300 Subject: [pytest-dev] document outlining added values In-Reply-To: <20181204121216.o3isvqmd35j3q2ex@hooch.localdomain> References:

<20181204121216.o3isvqmd35j3q2ex@hooch.localdomain> Message-ID: Wow, I sure wish I'd found that earlier. I swear I'd been looking for this for years before I started writing my own :) That said, I've learned an awful lot by writing my own; I wouldn't have been able to do the GDAL conversion without writing that first. Sivan, I'm sorry for misinterpreting your original post and (inadvertently) derailing your thread :) I'll try to answer the original question. We converted our unittest/django test suite to pytest style over the last few years (recently very quickly as a result of this conversion script). Here's a list of advantages we've seen. Some of these are to do with the pytest *runner* and some are to do with the particular style of pytest tests themselves. Syntactical advantages: - assertion syntax is more pythonic (less camelCase) - assertion syntax is less verbose, and equally if not more readable - assertion *messages* are rarely required with pytest, because it prints the sub-expressions of the expressions used in the assert statement. Function-based test advantages: - With a large enough test suite, each test case class becomes a woolly maze of multiple inheritance with mixins. We found this more confusing than the explicit dependency injection that pytest's fixtures offer - Having all functions in TestCase class objects makes it annoying to run a *particular* test; you have to type the classname as well as the module and test function name Performance: - Not *exactly* related but: Django's XML/JSON fixture files are terribly slow to load (or at least, were when I last checked, probably about django 1.8). XML was better than JSON but neither approach the speed you can get by creating objects manually via the ORM. Moving to pytest was a good opportunity to move these fixture files into pytest fixtures using the Django ORM. (Note: there are other ways to avoid this - you could use factory_boy or something and avoid django fixture files) Customisation: - We've found it much easier to extend the functionality of our test suite. For example, recently we added a new version to our API (all `/api/v1/endpoint/` things can be accessed via `/api/v2/endpoint/` too now). We trivially parametrized our test-client fixture, to run all API tests against both endpoints, and transparently rewrite the version numbers in the URLs. This wouldn't have been easily possible with a standard django test suite. - Pytest makes it easy to run fixtures with different scopes, whereas unittest-based testing assumes all setup happens for a particular class or function. In particular, session-scope fixtures allow us to do setup once for the whole test run. - It's easy to setup things *by default*. For example, we wanted to *remove* access to our storage volume during tests by default, so that we could verify that nothing touched it that didn't *need* to. This was easy with pytest - we just created an autouse fixture that runs before each test and ensures the storage volume is unavailable. The fixture is removed during `pytest_collection_modifyitems` if a particular marker is present to say a test needs storage access. - It's easy to customise what tests will be run, either with custom command-line flags, environment variables, or whatever you want. Just add a pytest_collection_modifyitems hook and tweak the list. You can skip tests, reorder tests, add fixtures, add markers... whatever you want. I don't know if this is even possible with the django test runner without heavy patching. I hope that's helpful. Switching to pytest has been a real breath of fresh air and has greatly accelerated our *desire* to write tests, which I think is important. It makes testing less of a drag, and maybe even fun :) Craig de Stigter Platform Engineer @ https://koordinates.com On Wed, 5 Dec 2018 at 01:12 Florian Bruhin wrote: > On Tue, Dec 04, 2018 at 12:54:50AM +0000, Craig de Stigter wrote: > > > gradually porting all tests > > > > Mostly-automated conversion is possible, so it doesn't have to be a > gradual > > process. > > > > I wrote a script which converts xunit-style assertions into pytest ones: > > https://github.com/craigds/decrapify/blob/master/pytestify.py > > Yet another one? ;-) > > https://github.com/pytest-dev/unittest2pytest > https://github.com/dropbox/dbx-unittest2pytest > > Florian > > -- > https://www.qutebrowser.org | me at the-compiler.org (Mail/XMPP) > GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc > I love long mails! | https://email.is-not-s.ms/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sivan at vitakka.co Tue Dec 4 16:12:51 2018 From: sivan at vitakka.co (=?UTF-8?Q?Sivan_Gr=C3=BCnberg?=) Date: Tue, 4 Dec 2018 23:12:51 +0200 Subject: [pytest-dev] document outlining added values In-Reply-To: References:

<20181204121216.o3isvqmd35j3q2ex@hooch.localdomain> Message-ID: No worries mate! It's nice to see that my simple question brought attention to two very cool plugins that assist in moving, and thank you @Florian Bruhin for popping in :) Thank you for the from-the-field experience account that's even better than a boiler plate comparison write up! I'll forward to my client so he can have this as a reference for the future when/if he decided porting is worthwhile. The 'derailing' of the thread actually showed me again a bit of the great community we have in. I so much regret not being able to attend that sprint in pretty Frieburg. -Sivan -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Sat Dec 8 20:42:35 2018 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sun, 9 Dec 2018 01:42:35 +0000 Subject: [pytest-dev] Using pytest for the sympy doctests Message-ID: Hi all, I'm trying to make it possible to use pytest with SymPy. I'm mostly there for the main tests now and the next step is to try and get it working for the doctests. Currently SymPy has it's own test runner which runs the doctests but it works a little differently from the standard doctests. Firstly SymPy's doctests are required to import all names use in the doctest. Standard doctest allows names from the enclosing module to be used without imports. It's considered important in SymPy that the doctests should error out if the imports are not included. So for example if I have def f(): '''' >>> f() 42 ''' return 42 then this should give an error in the doctests because the function f is undefined. A valid doctest for this would look like def f(): '''' >>> from module import f >>> f() 42 ''' return 42 It seems there is a feature in sympy to inject symbols into the doctest namespace: https://docs.pytest.org/en/latest/doctest.html#the-doctest-namespace-fixture but I'm really interested in removing things from the namespace to test for errors when something isn't properly imported in the doctest. The other issue is that SymPy has a few different ways of displaying output at the terminal: the repr, the ascii version, and the pretty version. The doctests are written for the ascii version but under pytest the pretty version is found as output. You can see this problem if you run e.g. $ git clone https://github.com/sympy/sympy.git $ cd sympy $ pytest --doctest-modules sympy/solvers/ode.py An example of the failures is /Users/enojb/current/sympy/sympy/sympy/solvers/ode.py:1501: DocTestFailure ___________________ [doctest] sympy.solvers.ode.constant_renumber 2896 Only constants in the given range (inclusive) are renumbered; 2897 the renumbering always starts from 1: 2898 2899 >>> constant_renumber(C1 + C3 + C4, 'C', 1, 3) 2900 C1 + C2 + C4 2901 >>> constant_renumber(C0 + C1 + C3 + C4, 'C', 2, 4) 2902 C0 + 2*C1 + C2 2903 >>> constant_renumber(C0 + 2*C1 + C2, 'C', 0, 1) 2904 C1 + 3*C2 2905 >>> pprint(C2 + C1*x + C3*x**2) Expected: 2 C1*x + C2 + C3*x Got: 2 C??x + C? + C??x To make this work I need to figure out a way for pytest to meet these two requirements: 1) Customise namespace used for tests 2) Customise the function that generates output before comparing with expected output of a doctest. Does anyone know if it's possible to do either of these things with pytest? -- Oscar From opensource at ronnypfannschmidt.de Sun Dec 9 03:47:29 2018 From: opensource at ronnypfannschmidt.de (Ronny Pfannschmidt) Date: Sun, 09 Dec 2018 09:47:29 +0100 Subject: [pytest-dev] Using pytest for the sympy doctests In-Reply-To: References: Message-ID: <0e542fd254159f3ad93662c2e9da3b529d07208c.camel@ronnypfannschmidt.de> Hi Oscar, the doctest plugin inside of pytest is mostly a barebones integration of the doctest module of the stdlib, adding the pytest extras by adding onto the default globals of doctest. As for the output variant, how does sympy decide which one to use, does the issue happen when the `-s` flag of pytest is used. my off hand my suggestion would be to make a external pytest-doctest2 plugin to experiment, once its solid i'd be more than happy to drop the doctest plugin in the core of pytest (as it really only deals with legacy ways of testing that are best migrated away from) -- Ronny Am Sonntag, den 09.12.2018, 01:42 +0000 schrieb Oscar Benjamin: > Hi all, > > I'm trying to make it possible to use pytest with SymPy. I'm mostly > there for the main tests now and the next step is to try and get it > working for the doctests. Currently SymPy has it's own test runner > which runs the doctests but it works a little differently from the > standard doctests. > > Firstly SymPy's doctests are required to import all names use in the > doctest. Standard doctest allows names from the enclosing module to > be > used without imports. It's considered important in SymPy that the > doctests should error out if the imports are not included. > > So for example if I have > > def f(): > '''' > >>> f() > 42 > ''' > return 42 > > then this should give an error in the doctests because the function f > is undefined. A valid doctest for this would look like > def f(): > '''' > >>> from module import f > >>> f() > 42 > ''' > return 42 > > It seems there is a feature in sympy to inject symbols into the > doctest namespace: > https://docs.pytest.org/en/latest/doctest.html#the-doctest-namespace-fixture > but I'm really interested in removing things from the namespace to > test for errors when something isn't properly imported in the > doctest. > > The other issue is that SymPy has a few different ways of displaying > output at the terminal: the repr, the ascii version, and the pretty > version. The doctests are written for the ascii version but under > pytest the pretty version is found as output. You can see this > problem > if you run e.g. > > $ git clone https://github.com/sympy/sympy.git > $ cd sympy > $ pytest --doctest-modules sympy/solvers/ode.py > > An example of the failures is > > /Users/enojb/current/sympy/sympy/sympy/solvers/ode.py:1501: > DocTestFailure > ___________________ [doctest] sympy.solvers.ode.constant_renumber > 2896 Only constants in the given range (inclusive) are > renumbered; > 2897 the renumbering always starts from 1: > 2898 > 2899 >>> constant_renumber(C1 + C3 + C4, 'C', 1, 3) > 2900 C1 + C2 + C4 > 2901 >>> constant_renumber(C0 + C1 + C3 + C4, 'C', 2, 4) > 2902 C0 + 2*C1 + C2 > 2903 >>> constant_renumber(C0 + 2*C1 + C2, 'C', 0, 1) > 2904 C1 + 3*C2 > 2905 >>> pprint(C2 + C1*x + C3*x**2) > Expected: > 2 > C1*x + C2 + C3*x > Got: > 2 > C??x + C? + C??x > > To make this work I need to figure out a way for pytest to meet these > two requirements: > 1) Customise namespace used for tests > 2) Customise the function that generates output before comparing with > expected output of a doctest. > > Does anyone know if it's possible to do either of these things with > pytest? > > -- > Oscar > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev From oscar.j.benjamin at gmail.com Sun Dec 9 09:30:29 2018 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sun, 9 Dec 2018 14:30:29 +0000 Subject: [pytest-dev] Using pytest for the sympy doctests In-Reply-To: <0e542fd254159f3ad93662c2e9da3b529d07208c.camel@ronnypfannschmidt.de> References: <0e542fd254159f3ad93662c2e9da3b529d07208c.camel@ronnypfannschmidt.de> Message-ID: On Sun, 9 Dec 2018 at 08:47, Ronny Pfannschmidt wrote: > > As for the output variant, how does sympy decide which one to use, > does the issue happen when the `-s` flag of pytest is used. Yes it does but actually this is a good point. The pprint function being used there is sympy's own pprint and the existing doctest runner has some code to modify its behaviour. That looks like it might be fixable within sympy then. I added from sympy.utilities.runtests import setup_pprint setup_pprint() to conftest.py and created a pytest.ini with [pytest] doctest_optionflags= NORMALIZE_WHITESPACE IGNORE_EXCEPTION_DETAIL and that's solved most of the failures. Is there any way to set those doctest option flags in conftest.py rather than pytest.ini? I am still getting a few errors like this from the repr though: ______________ [doctest] sympy.solvers.ode._undetermined_coefficients_match ... 5228 >>> _undetermined_coefficients_match(9*x*exp(x) + exp(-x), x) Expected: {'test': True, 'trialset': {x*exp(x), exp(-x), exp(x)}} Got: {'test': True, 'trialset': {exp(x), exp(-x), x*exp(x)}} This is somehow handled in sympy's test runner using sys.displayhook. If I can track that down in sympy it might be possible to solve these as well. > my off hand my suggestion would be to make a external pytest-doctest2 > plugin to experiment, once its solid i'd be more than happy to drop the > doctest plugin in the core of pytest (as it really only deals with > legacy ways of testing that are best migrated away from) That could be a good idea. Even if pytest didn't want to merge it sympy's test code could become a pytest plugin rather than its own reimplementation of pytest. I'll see how far I get with conftest.py first though... -- Oscar From nicoddemus at gmail.com Sun Dec 9 11:31:21 2018 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Sun, 9 Dec 2018 14:31:21 -0200 Subject: [pytest-dev] Using pytest for the sympy doctests In-Reply-To: References: <0e542fd254159f3ad93662c2e9da3b529d07208c.camel@ronnypfannschmidt.de> Message-ID: Hi Oscar, On Sun, Dec 9, 2018 at 12:30 PM Oscar Benjamin wrote: > Is there any way to set those doctest option flags in conftest.py > rather than pytest.ini? > Currently those options are only possible to be set in pytest.ini, I'm afraid. Any reason why is that a problem though? Cheers, Bruno -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Sun Dec 9 13:35:29 2018 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sun, 9 Dec 2018 18:35:29 +0000 Subject: [pytest-dev] Using pytest for the sympy doctests In-Reply-To: References: <0e542fd254159f3ad93662c2e9da3b529d07208c.camel@ronnypfannschmidt.de>

Message-ID: On Sun, 9 Dec 2018 at 16:31, Bruno Oliveira wrote: > > Hi Oscar, Hi Bruno, > On Sun, Dec 9, 2018 at 12:30 PM Oscar Benjamin wrote: >> >> Is there any way to set those doctest option flags in conftest.py >> rather than pytest.ini? > > Currently those options are only possible to be set in pytest.ini, I'm afraid. Okay, thanks for letting me know. > Any reason why is that a problem though? It's not a huge problem. If it were possible to set them in conftest.py I'd rather do that because the pytest.ini doesn't exist yet and I'd like to keep everything in one file if possible. I spent a little time trying to work out if was possible: I just imagined there would be a generic way to set ini options from the conftest.py file. It's not needed in this case but the advantage of being able to set those things in conftest.py could be if you want to make them conditional on something. Thanks, Oscar From nicoddemus at gmail.com Sun Dec 9 14:55:55 2018 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Sun, 9 Dec 2018 17:55:55 -0200 Subject: [pytest-dev] Using pytest for the sympy doctests In-Reply-To: References: <0e542fd254159f3ad93662c2e9da3b529d07208c.camel@ronnypfannschmidt.de>

Message-ID: On Sun, Dec 9, 2018 at 4:35 PM Oscar Benjamin wrote: > It's not a huge problem. If it were possible to set them in > conftest.py I'd rather do that because the pytest.ini doesn't exist > yet and I'd like to keep everything in one file if possible. > > I spent a little time trying to work out if was possible: I just > imagined there would be a generic way to set ini options from the > conftest.py file. It's not needed in this case but the advantage of > being able to set those things in conftest.py could be if you want to > make them conditional on something. > Sure, makes sense. We could add a new "config.setini" option, for one it would allow users to set ini values in pytest_configure, I remember needing this in more than one occasion actually. There's a discussion about this in https://github.com/pytest-dev/pytest/issues/3311, in there the OP wanted to do this so plugins could change ini options from other plugins (all under his control) so it was considered a little brittle, but I think the use case for changing ini variables during pytest_configure like you describe is a strong motivation for us to add this API. Ronny, what's your opinion? Cheers, Bruno. -------------- next part -------------- An HTML attachment was scrubbed... URL: From opensource at ronnypfannschmidt.de Sun Dec 9 15:27:25 2018 From: opensource at ronnypfannschmidt.de (Ronny Pfannschmidt) Date: Sun, 09 Dec 2018 21:27:25 +0100 Subject: [pytest-dev] Using pytest for the sympy doctests In-Reply-To: References: <0e542fd254159f3ad93662c2e9da3b529d07208c.camel@ronnypfannschmidt.de>

Message-ID: Am Sonntag, den 09.12.2018, 17:55 -0200 schrieb Bruno Oliveira: > On Sun, Dec 9, 2018 at 4:35 PM Oscar Benjamin < > oscar.j.benjamin at gmail.com> wrote: > > It's not a huge problem. If it were possible to set them in > > > > conftest.py I'd rather do that because the pytest.ini doesn't exist > > > > yet and I'd like to keep everything in one file if possible. > > > > > > > > I spent a little time trying to work out if was possible: I just > > > > imagined there would be a generic way to set ini options from the > > > > conftest.py file. It's not needed in this case but the advantage of > > > > being able to set those things in conftest.py could be if you want > > to > > > > make them conditional on something. > > Sure, makes sense. We could add a new "config.setini" option, for one > it would allow users to set ini values in pytest_configure, I > remember needing this in more than one occasion actually. > > There's a discussion about this in > https://github.com/pytest-dev/pytest/issues/3311, in there the OP > wanted to do this so plugins could change ini options from other > plugins (all under his control) so it was considered a little > brittle, but I think the use case for changing ini variables during > pytest_configure like you describe is a strong motivation for us to > add this API. > > Ronny, what's your opinion? my personal preference is to remove support for programatic changing of options with the config object - it makes things horrendous for various reasons,even addinilines is one too much -- Ronny > Cheers, > Bruno. > > _______________________________________________pytest-dev mailing > listpytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicoddemus at gmail.com Sun Dec 9 15:34:59 2018 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Sun, 9 Dec 2018 18:34:59 -0200 Subject: [pytest-dev] Using pytest for the sympy doctests In-Reply-To: References: <0e542fd254159f3ad93662c2e9da3b529d07208c.camel@ronnypfannschmidt.de>

Message-ID: On Sun, Dec 9, 2018 at 6:27 PM Ronny Pfannschmidt < opensource at ronnypfannschmidt.de> wrote: > Am Sonntag, den 09.12.2018, 17:55 -0200 schrieb Bruno Oliveira: > > > > On Sun, Dec 9, 2018 at 4:35 PM Oscar Benjamin > wrote: > > It's not a huge problem. If it were possible to set them in > conftest.py I'd rather do that because the pytest.ini doesn't exist > yet and I'd like to keep everything in one file if possible. > > I spent a little time trying to work out if was possible: I just > imagined there would be a generic way to set ini options from the > conftest.py file. It's not needed in this case but the advantage of > being able to set those things in conftest.py could be if you want to > make them conditional on something. > > > Sure, makes sense. We could add a new "config.setini" option, for one it > would allow users to set ini values in pytest_configure, I remember needing > this in more than one occasion actually. > > There's a discussion about this in > https://github.com/pytest-dev/pytest/issues/3311, in there the OP wanted > to do this so plugins could change ini options from other plugins (all > under his control) so it was considered a little brittle, but I think the > use case for changing ini variables during pytest_configure like you > describe is a strong motivation for us to add this API. > > Ronny, what's your opinion? > > > my personal preference is to remove support for programatic changing of > options with the config object - it makes things horrendous for various > reasons, > even addinilines is one too much > I see, could you enumerate some of those reasons please? It would help for us to understand better the underlying problems with that approach. Perhaps one of the reasons is that the order in which the ini variables are registered is not well defined? Cheers, Bruno. > -- Ronny > > > Cheers, > Bruno. > > _______________________________________________ > > pytest-dev mailing list > > pytest-dev at python.org > > > https://mail.python.org/mailman/listinfo/pytest-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sylvain.marie at se.com Wed Dec 12 10:31:34 2018 From: sylvain.marie at se.com (Sylvain MARIE) Date: Wed, 12 Dec 2018 15:31:34 +0000 Subject: [pytest-dev] data science benchmarks with pytest Message-ID: Dear pytest community I am pleased to share with you today the first example in https://smarie.github.io/pytest-patterns/ ! This first example shows how to create a data science benchmark, like the ones in kaggle or driven data, using pytest and a few plugins (you will recognize the names from my previous posts ;) ). Maybe in the future someone in the community will find other ?best practices? to share in terms of parameters/fixtures architectures for specific applications. In that case this project can serve as a placeholder for such examples. Kind regards -- Sylvain -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicoddemus at gmail.com Fri Dec 14 05:29:26 2018 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Fri, 14 Dec 2018 08:29:26 -0200 Subject: [pytest-dev] pytest 4.0.2 Message-ID: Hi everyone, pytest 4.0.2 has just been released to PyPI. This is a bug-fix release, being a drop-in replacement. To upgrade:: pip install --upgrade pytest The full changelog is available at https://docs.pytest.org/en/latest/changelog.html. Thanks to all who contributed to this release, among them: * Anthony Sottile * Bruno Oliveira * Daniel Hahler * Pedro Algarvio * Ronny Pfannschmidt * Tomer Keren * Yash Todi Happy testing, The pytest Development Team -------------- next part -------------- An HTML attachment was scrubbed... URL: