From nicoddemus at gmail.com Wed Oct 4 16:32:11 2017 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Wed, 04 Oct 2017 20:32:11 +0000 Subject: [pytest-dev] Pytest 3.2.3 released Message-ID: Howdy everyone, pytest 3.2.3 has just been released to PyPI! This is a bug-fix release, being a drop-in replacement. To upgrade:: pip install --upgrade pytest The full changelog is available at http://doc.pytest.org/en/latest/changelog.html. Thanks to all who contributed to this release, among them: * Bruno Oliveira * Evan * Joe Hamman * Oliver Bestwalter * Ronny Pfannschmidt * Xuan Luong Happy testing, The pytest Development Team -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan.nealschneider at gmail.com Sat Oct 7 16:32:55 2017 From: dan.nealschneider at gmail.com (Dan Nealschneider) Date: Sat, 7 Oct 2017 13:32:55 -0700 Subject: [pytest-dev] Garbage collection and pytest fixtures Message-ID: Hey all- I've been exploring some of our tests that use a lot of memory, and whose memory use seems to grow over the course of a test run. In my preliminary investiagtion it seems like some of the memory use can be attributed to fixtures which create large objects. I'm using py.test 3.1.3 in python 2.7.13. We have a bunch of conftests and a local pytest plugin that changes some behaviors, so I want to check in about intended behavior before I go further. If I have a fixture like the following, should I expect the object returned by the fixture to be garbage collected after the tests that use it complete? Or after all the tests in the module complete? Or after the seesion completes? @pytest.fixture(scope='module') def big_item(): return [0] *1000000 Is this a reasonable way to use a fixture, or is there a better way to reuse a large and expensive-to-create item across tests? Thanks, Dan N -------------- next part -------------- An HTML attachment was scrubbed... URL: From flub at devork.be Sun Oct 8 16:03:55 2017 From: flub at devork.be (Floris Bruynooghe) Date: Sun, 8 Oct 2017 22:03:55 +0200 Subject: [pytest-dev] Garbage collection and pytest fixtures In-Reply-To: References: Message-ID: Hi Dan, On 7 October 2017 at 22:32, Dan Nealschneider wrote: > If I have a fixture like the following, should I expect the object returned > by the fixture to be garbage collected after the tests that use it complete? > Or after all the tests in the module complete? Or after the seesion > completes? > > @pytest.fixture(scope='module') > def big_item(): > return [0] *1000000 > > Is this a reasonable way to use a fixture, or is there a better way to reuse > a large and > expensive-to-create item across tests? This is perfectly reasonable. The fixture should be torn down soon after the last test uses it and then it should be up to garbage collection to clean it up. In the worst case this would indeed stay in memory for the whole module. If pytest somehow keeps references after this that's probably a bug. Note however that there is one exception to this. If any of the tests with the fixture failed then the traceback object of that failed test will stay in memory for pytest, so that it can use this later for reporting etc. If this traceback somehow refers to the large fixture then it will be kept alive. If this really is a problem then there's probably some work that could be done, but it won't be trivial and will need someone to dedicate some time to improve this. I think this has been mentioned in the past however and in the end no one was hurt that bad by it that they figured out how to improve this. Kind regards, Floris From opensource at ronnypfannschmidt.de Wed Oct 25 03:54:56 2017 From: opensource at ronnypfannschmidt.de (RonnyPfannschmidt) Date: Wed, 25 Oct 2017 09:54:56 +0200 Subject: [pytest-dev] deprecating `pytest_plugins` in non-initial conftests to prevent unintended smear/spill of plugin configurations Message-ID: hi everyone, while reviewing the codebase i noticed, that the code that is ensuring we don't use conftests from other folders don't take into account `pytest_plugins` declared in such a conftest thus the plugin system gives a false impression. intuitively i'd expect plugins listed in a conftest only to be active when that conftest is active however due to the way our plugin system works, we only mark conftests of other folders as "disabled" not their dependencies i dont see a sane way to implement this feature correctly, as we wouldnt be allowed to disable a plugin just because its listed in one folder, after all it may be listed in multiple folders but not one certain folder the plugin system as we constructed is is not aware of hierarchical dependencies and i dont think we can get this good/right with the current development process we use as such i propose deprecating them and warning users about the problems with regard to when and how to warn we need to take a look in detail as there are many reasonably valid setups with exactly one non initial conftest and we should help people to turn those into initial conftest's while cleaning out the rest. -- Ronny From flub at devork.be Wed Oct 25 16:01:13 2017 From: flub at devork.be (Floris Bruynooghe) Date: Wed, 25 Oct 2017 22:01:13 +0200 Subject: [pytest-dev] deprecating `pytest_plugins` in non-initial conftests to prevent unintended smear/spill of plugin configurations In-Reply-To: References: Message-ID: On 25 October 2017 at 09:54, RonnyPfannschmidt wrote: > hi everyone, > > while reviewing the codebase i noticed, that the code that is ensuring > we don't use conftests from other folders don't take into account > `pytest_plugins` declared in such a conftest > thus the plugin system gives a false impression. I think I've always advised against using pytest_plugins in anything else but the toplevel conftest.py. I thought this was the de-facto rule. > intuitively i'd expect plugins listed in a conftest only to be active > when that conftest is active That seems somewhat twisted to me, not sure I like your intuition, but I can see how you get there ;-) > as such i propose deprecating them and warning users about the problems > with regard to when and how to warn we need to take a look in detail as > there are many reasonably valid setups with exactly one non initial conftest > and we should help people to turn those into initial conftest's while > cleaning out the rest. Absolutely, if we can detect this and warn users away from it then that sounds great. It could go on the "removal in 4.0" list if such a list existed. Cheers, Floris From nicoddemus at gmail.com Wed Oct 25 17:25:34 2017 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Wed, 25 Oct 2017 21:25:34 +0000 Subject: [pytest-dev] deprecating `pytest_plugins` in non-initial conftests to prevent unintended smear/spill of plugin configurations In-Reply-To: References: Message-ID: On Wed, Oct 25, 2017 at 6:01 PM Floris Bruynooghe wrote: > > as such i propose deprecating them and warning users about the problems > > with regard to when and how to warn we need to take a look in detail as > > there are many reasonably valid setups with exactly one non initial > conftest > > and we should help people to turn those into initial conftest's while > > cleaning out the rest. > > Absolutely, if we can detect this and warn users away from it then > that sounds great. It could go on the "removal in 4.0" list if such a > list existed. > Agreed. We should be able to detect a pytest_plugins directive in non-initial conftest files fairly easily and issue a warning if that's the case. We have a deprecation list: https://github.com/pytest-dev/pytest/wiki/Deprecation-Roadmap Cheers, Bruno -------------- next part -------------- An HTML attachment was scrubbed... URL: From opensource at ronnypfannschmidt.de Fri Oct 27 12:57:42 2017 From: opensource at ronnypfannschmidt.de (RonnyPfannschmidt) Date: Fri, 27 Oct 2017 18:57:42 +0200 Subject: [pytest-dev] proposal - type-keyed storage for extension objects/metadata in config/items Message-ID: <4b79752c-495b-4e8a-fac1-cd37b994eb27@ronnypfannschmidt.de> hi everyone since a while now i feel the lack of a place where i can store objects that belong to some pytest plugin but shouldn't be plugins themselves i came up with an idea for a basic api and would like to get feedback from it at the basic level there would be a `config.objects` or `item.objects` as "immutable" mapping with 4 extra methods to mutate, i#ll just show the proposed effects get_or_create(tp, *k, **kw): ? if tp not in self: ??? self[tp] = tp(*k, **kw) ? return self[tp] create(self, tp, *k, **kw): ? if tp in self: ??? raise KeyError(tp) ?self[tp] = tp(*k, **kw) ?return self.tp ?add(obj): ? tp = type(obj) ? if tp in self: ??? raise KeyError(tp) ? self[tp] = obj remove(self, obj): ? del self[type(obj)] discard(self, obj): ? self.pop(type(obj), None) by adding such a storage it will be a lot more easy to just put things on items and reports -- Ronny -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicoddemus at gmail.com Fri Oct 27 13:55:05 2017 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Fri, 27 Oct 2017 17:55:05 +0000 Subject: [pytest-dev] proposal - type-keyed storage for extension objects/metadata in config/items In-Reply-To: <4b79752c-495b-4e8a-fac1-cd37b994eb27@ronnypfannschmidt.de> References: <4b79752c-495b-4e8a-fac1-cd37b994eb27@ronnypfannschmidt.de> Message-ID: Hi Ronny, Can you provide a use case for this storage? Perhaps we can use existing cases and see how they would like with the new API. For example, pytest-html currently attaches an "extra" dictionary to items. How could pytest-html make use of the new API? Another example is the core junitxml plugin, which adds new items that should go into the reporty by using a fixture. How it would look like to customize the report using the new API? Cheers, Bruno. On Fri, Oct 27, 2017 at 2:58 PM RonnyPfannschmidt < opensource at ronnypfannschmidt.de> wrote: > hi everyone > > since a while now i feel the lack of a place where i can store objects > that belong to some pytest plugin but shouldn't be plugins themselves > > i came up with an idea for a basic api and would like to get feedback > from it > > at the basic level there would be a `config.objects` or `item.objects` > as "immutable" mapping with 4 extra methods to mutate, i#ll just show > the proposed effects > > get_or_create(tp, *k, **kw): > if tp not in self: > self[tp] = tp(*k, **kw) > return self[tp] > create(self, tp, *k, **kw): > if tp in self: > raise KeyError(tp) > self[tp] = tp(*k, **kw) > return self.tp > > add(obj): > tp = type(obj) > if tp in self: > raise KeyError(tp) > self[tp] = obj > > remove(self, obj): > del self[type(obj)] > > discard(self, obj): > self.pop(type(obj), None) > > by adding such a storage it will be a lot more easy to just put things > on items and reports > > > -- Ronny > > > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tgoodlet at gmail.com Fri Oct 27 14:30:59 2017 From: tgoodlet at gmail.com (Tyler Goodlet) Date: Fri, 27 Oct 2017 14:30:59 -0400 Subject: [pytest-dev] proposal - type-keyed storage for extension objects/metadata in config/items In-Reply-To: References: <4b79752c-495b-4e8a-fac1-cd37b994eb27@ronnypfannschmidt.de> Message-ID: >Can you provide a use case for this storage? Perhaps we can use existing cases and see how they would like with the new API. ^ I agree it'd be good to see a practical example. I'm guessing this is to address what the ``pytest_namespace`` stuff (at least in the `config.objects` case) used to fulfil? I know that with a project I've been meaning to get back to (`pytestlab` or whatever it turns into) we wanted to solve this problem but for managing testing resources and environments (eg here: https://github.com/sangoma/pytestlab/blob/master/pytest_lab/map.py#L13). I know for the purpose of system testing other full services using pytest we wanted to have a data store that was network transparent as well to meet the needs of distributed test runs that require synchronization and other protocols between inter-test stages. Having a base system + API in pytest core would make building a solution to such problems that much easier. Anyway just thinking out loud. Tyler Goodlet On Fri, Oct 27, 2017 at 1:55 PM, Bruno Oliveira wrote: > Hi Ronny, > > Can you provide a use case for this storage? Perhaps we can use existing > cases and see how they would like with the new API. > > For example, pytest-html currently attaches an "extra" dictionary to > items. How could pytest-html make use of the new API? > > Another example is the core junitxml plugin, which adds new items that > should go into the reporty by using a fixture. How it would look like to > customize the report using the new API? > > Cheers, > Bruno. > > On Fri, Oct 27, 2017 at 2:58 PM RonnyPfannschmidt < > opensource at ronnypfannschmidt.de> wrote: > >> hi everyone >> >> since a while now i feel the lack of a place where i can store objects >> that belong to some pytest plugin but shouldn't be plugins themselves >> >> i came up with an idea for a basic api and would like to get feedback >> from it >> >> at the basic level there would be a `config.objects` or `item.objects` >> as "immutable" mapping with 4 extra methods to mutate, i#ll just show >> the proposed effects >> >> get_or_create(tp, *k, **kw): >> if tp not in self: >> self[tp] = tp(*k, **kw) >> return self[tp] >> create(self, tp, *k, **kw): >> if tp in self: >> raise KeyError(tp) >> self[tp] = tp(*k, **kw) >> return self.tp >> >> add(obj): >> tp = type(obj) >> if tp in self: >> raise KeyError(tp) >> self[tp] = obj >> >> remove(self, obj): >> del self[type(obj)] >> >> discard(self, obj): >> self.pop(type(obj), None) >> >> by adding such a storage it will be a lot more easy to just put things >> on items and reports >> >> >> -- Ronny >> >> >> _______________________________________________ >> pytest-dev mailing list >> pytest-dev at python.org >> https://mail.python.org/mailman/listinfo/pytest-dev >> > > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: