From steve at lonetwin.net  Mon Feb  6 10:56:33 2023
From: steve at lonetwin.net (Steve)
Date: Mon, 06 Feb 2023 15:56:33 +0000
Subject: [pytest-dev] Looking for recommendation
Message-ID: <1770841517943c4eb725eb13447283cf718bbdb1.camel@lonetwin.net>

Hello,

I'm look for some advice regarding restructuring tests/test layout for an existing project using pytest.

Forgive me if this isn't the correct forum for this sort of questions. I'm sending this here because I felt this doesn't
really qualify as a bug/issue to be raise on github.

Here's what I am looking for -- I'd like to update / restructure test within an existing project which initially was
developed as a webapp for a single client/customer but is now being refactored to allow for a multi-client / multi-
tenent setup.

The current tests layout is pretty standard with the tests/ directory living next to the src/ directory and a top-level
conftest.py with fixtures. 

In the new version of the app, we would like to have tests that are:

a. Common for all deployments and are entirely independent of customer configuration
b. Common tests per customer configuration (potentially using parameterised fixtures and/or tests) 
c. Customer specific tests (potentially using markers)

I am curious whether there are any best-practices I could adopt and pitfalls I ought to be aware of.

Additionally, any recommendations on tests directory layout and fixture approaches would also help.

Ideally, the setup should be flexible enough so that we might to exercise all the tests (for all customers) or select
all the common tests + parameterized tests + customer specific tests for a specific customer.

I'm considering a combination of using custom commandline options[1] and markers to do this.

Does a structure like...

tests
  |- conftest.py
  |- tests_common/
  |    |- tests_parameterized.py
  !    |- tests_independent.py
  |- customer1/
  |    |- conftest.py
  |    |- tests_speciic.py
  |- customer2/
  |- ...

make sense ?

I know this is all a bit high-level and vague but I'm trying to learn from any of you who might have done this before.
So, I recognise and appreciate in advance any time you put into replying to this.

cheers,
Steve


[1] https://docs.pytest.org/en/7.2.x/example/simple.html#control-skipping-of-tests-according-to-command-line-option

From nicoddemus at gmail.com  Mon Feb  6 11:27:29 2023
From: nicoddemus at gmail.com (Bruno Oliveira)
Date: Mon, 6 Feb 2023 13:27:29 -0300
Subject: [pytest-dev] Looking for recommendation
In-Reply-To: <1770841517943c4eb725eb13447283cf718bbdb1.camel@lonetwin.net>
References: <1770841517943c4eb725eb13447283cf718bbdb1.camel@lonetwin.net>
Message-ID: <CA+RQFff6KpBoRHSetMZHNrEBoOUJzrBf9_i_3Gt3b5s6BKg4ow@mail.gmail.com>

Hi Steve,

One technique which I have used for this scenario is to declare a base
class containing the "common" tests. The class should not have the "Test"
prefix/suffix as to not be collected by pytest.

Then on the sibling projects, you create a Test subclass from the base
class, which in turns makes it collectible by pytest. One point you want to
address is how to customize the tests somehow on the sibling project. If
you are already using a fixture based system, it is easy to overwrite the
fixture in the subclass to return project-specific data which makes sense
in that context.

I describe this technique in detail in my pytest book, pytest Quickstart
Guide if you want more details.

Hope this points you in the right direction.

Cheers,
Bruno.

On Mon, Feb 6, 2023 at 1:21 PM Steve <steve at lonetwin.net> wrote:

> Hello,
>
> I'm look for some advice regarding restructuring tests/test layout for an
> existing project using pytest.
>
> Forgive me if this isn't the correct forum for this sort of questions. I'm
> sending this here because I felt this doesn't
> really qualify as a bug/issue to be raise on github.
>
> Here's what I am looking for -- I'd like to update / restructure test
> within an existing project which initially was
> developed as a webapp for a single client/customer but is now being
> refactored to allow for a multi-client / multi-
> tenent setup.
>
> The current tests layout is pretty standard with the tests/ directory
> living next to the src/ directory and a top-level
> conftest.py with fixtures.
>
> In the new version of the app, we would like to have tests that are:
>
> a. Common for all deployments and are entirely independent of customer
> configuration
> b. Common tests per customer configuration (potentially using
> parameterised fixtures and/or tests)
> c. Customer specific tests (potentially using markers)
>
> I am curious whether there are any best-practices I could adopt and
> pitfalls I ought to be aware of.
>
> Additionally, any recommendations on tests directory layout and fixture
> approaches would also help.
>
> Ideally, the setup should be flexible enough so that we might to exercise
> all the tests (for all customers) or select
> all the common tests + parameterized tests + customer specific tests for a
> specific customer.
>
> I'm considering a combination of using custom commandline options[1] and
> markers to do this.
>
> Does a structure like...
>
> tests
>   |- conftest.py
>   |- tests_common/
>   |    |- tests_parameterized.py
>   !    |- tests_independent.py
>   |- customer1/
>   |    |- conftest.py
>   |    |- tests_speciic.py
>   |- customer2/
>   |- ...
>
> make sense ?
>
> I know this is all a bit high-level and vague but I'm trying to learn from
> any of you who might have done this before.
> So, I recognise and appreciate in advance any time you put into replying
> to this.
>
> cheers,
> Steve
>
>
> [1]
> https://docs.pytest.org/en/7.2.x/example/simple.html#control-skipping-of-tests-according-to-command-line-option
> _______________________________________________
> pytest-dev mailing list
> pytest-dev at python.org
> https://mail.python.org/mailman/listinfo/pytest-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pytest-dev/attachments/20230206/b0342060/attachment.html>

From steve at lonetwin.net  Mon Feb  6 11:55:09 2023
From: steve at lonetwin.net (Steve)
Date: Mon, 06 Feb 2023 16:55:09 +0000
Subject: [pytest-dev] Looking for recommendation
In-Reply-To: <CA+RQFff6KpBoRHSetMZHNrEBoOUJzrBf9_i_3Gt3b5s6BKg4ow@mail.gmail.com>
References: <1770841517943c4eb725eb13447283cf718bbdb1.camel@lonetwin.net>
 <CA+RQFff6KpBoRHSetMZHNrEBoOUJzrBf9_i_3Gt3b5s6BKg4ow@mail.gmail.com>
Message-ID: <8d62d32156844743756bc756a9d04f9c0fb37355.camel@lonetwin.net>

Hello Bruno,

Thanks for the insightful reply.

That's a neat trick -- depending on pytest's class name collection approach to create a hierarchy for tests.

This approach addresses both, the sharing of tests/fixtures as well as having custom tests/fixtures.
It might also be possible to be 'smart' w.r.t the parameters in parameterised tests.

I'm going through your book now.

Thanks again,
Steve


On Mon, 2023-02-06 at 13:27 -0300, Bruno Oliveira wrote:
> Hi Steve,
> 
> One technique which I have used for this scenario is to declare a base class containing the "common" tests. The class
> should not have the "Test" prefix/suffix as to not be collected by pytest.
> 
> Then on the sibling projects, you create a Test subclass from the base class, which in turns makes it collectible by
> pytest. One point you want to address is how to customize the tests somehow on the sibling project. If you are already
> using a fixture based system, it is easy to overwrite the fixture in the subclass to return project-specific data
> which?makes sense in that context.
> 
> I describe this technique in detail in my pytest book, pytest Quickstart Guide if you want more details.
> 
> Hope this points you in the right direction.
> 
> Cheers,
> Bruno.
> 
> On Mon, Feb 6, 2023 at 1:21 PM Steve <steve at lonetwin.net> wrote:
> > Hello,
> > 
> > I'm look for some advice regarding restructuring tests/test layout for an existing project using pytest.
> > 
> > Forgive me if this isn't the correct forum for this sort of questions. I'm sending this here because I felt this
> > doesn't
> > really qualify as a bug/issue to be raise on github.
> > 
> > Here's what I am looking for -- I'd like to update / restructure test within an existing project which initially was
> > developed as a webapp for a single client/customer but is now being refactored to allow for a multi-client / multi-
> > tenent setup.
> > 
> > The current tests layout is pretty standard with the tests/ directory living next to the src/ directory and a top-
> > level
> > conftest.py with fixtures. 
> > 
> > In the new version of the app, we would like to have tests that are:
> > 
> > a. Common for all deployments and are entirely independent of customer configuration
> > b. Common tests per customer configuration (potentially using parameterised fixtures and/or tests) 
> > c. Customer specific tests (potentially using markers)
> > 
> > I am curious whether there are any best-practices I could adopt and pitfalls I ought to be aware of.
> > 
> > Additionally, any recommendations on tests directory layout and fixture approaches would also help.
> > 
> > Ideally, the setup should be flexible enough so that we might to exercise all the tests (for all customers) or
> > select
> > all the common tests + parameterized tests + customer specific tests for a specific customer.
> > 
> > I'm considering a combination of using custom commandline options[1] and markers to do this.
> > 
> > Does a structure like...
> > 
> > tests
> > ? |- conftest.py
> > ? |- tests_common/
> > ? |? ? |- tests_parameterized.py
> > ? !? ? |- tests_independent.py
> > ? |- customer1/
> > ? |? ? |- conftest.py
> > ? |? ? |- tests_speciic.py
> > ? |- customer2/
> > ? |- ...
> > 
> > make sense ?
> > 
> > I know this is all a bit high-level and vague but I'm trying to learn from any of you who might have done this
> > before.
> > So, I recognise and appreciate in advance any time you put into replying to this.
> > 
> > cheers,
> > Steve
> > 
> > 
> > [1] https://docs.pytest.org/en/7.2.x/example/simple.html#control-skipping-of-tests-according-to-command-line-option
> > _______________________________________________
> > pytest-dev mailing list
> > pytest-dev at python.org
> > https://mail.python.org/mailman/listinfo/pytest-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pytest-dev/attachments/20230206/d2bccca0/attachment.html>

From chethana at pycascades.com  Fri Feb 10 19:28:08 2023
From: chethana at pycascades.com (Chethana Gopinath)
Date: Fri, 10 Feb 2023 16:28:08 -0800
Subject: [pytest-dev] Invitation to Sprints day @ Pycascades '23
Message-ID: <CAGXwL+zYrJFV_86a-3y2HWL-S=u5kVe3jszKdXXD3a2rcaLWiA@mail.gmail.com>

Hello!

My name is Chethana and I?m the Sprints Chair at Pycascades '23.

We came across Pytest <https://github.com/pytest-dev/pytest> and felt that
it would add immense value to our participants and allow for them to learn
more about tools to simplify writing tests, how to implement that
functionality within a tool and also provide them with exposure into open
source software and its processes.

So I wanted to reach out and see if you would be interested in
participating with your project at Sprints
<https://2023.pycascades.com/program/sprints/> at Pycascades this year.

If you would like to participate, please reply to this email directly, fill
out this interest form <https://bit.ly/PyCascades2023SprintIntakeForm> or
contact us at sprints at pycascades.com.

Looking forward to hearing from you,

Thank you!

-- 
*Chethana Gopinath*
Sprints Chair, PyCascades 2023
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pytest-dev/attachments/20230210/b97d2915/attachment.html>

From felipecrp at gmail.com  Fri Feb 24 09:17:37 2023
From: felipecrp at gmail.com (Felipe Curty)
Date: Fri, 24 Feb 2023 11:17:37 -0300
Subject: [pytest-dev] Help creating plugin that changes output
Message-ID: <CAA1VmgVu8fB9DzRhfXmRZ6p0aT8qV-ku+6BCQPt=3sZoa4vtxg@mail.gmail.com>

Hello,

I have been using pytest for a while and want to write a plugin to change
its output.

I would like to achieve the following output:

From:

test/test_sample.py .

To

test/test_sample.py
A House
 with two floors
   has stairs                [Passed]

I intend to use pytest_runtest_logreport in the setup phase to print the "A
House with two floors has stairs". And to use pytest_report_teststatus to
print the [Passed]. However, in `pytest_runtest_logreport`, I do not have
access to TerminalReporter. Hence, apparently, the only alternative is to
use the `print` command. Is there any problem using print instead of
Terminal Reporter?

I checked that some plugins already change the output, but they do this by
changing the TerminalReporter class. For instance, pydev-it creates a
subclass of the TerminalReporter (which is now a @final class and should
not be extended), and pydev-spec patches some methods of the
TerminalReporter.

Regards,
Felipe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pytest-dev/attachments/20230224/b701e910/attachment.html>

From felipecrp at gmail.com  Fri Feb 24 18:23:27 2023
From: felipecrp at gmail.com (Felipe Curty)
Date: Fri, 24 Feb 2023 20:23:27 -0300
Subject: [pytest-dev] Help creating plugin that changes output
In-Reply-To: <CAA1VmgVu8fB9DzRhfXmRZ6p0aT8qV-ku+6BCQPt=3sZoa4vtxg@mail.gmail.com>
References: <CAA1VmgVu8fB9DzRhfXmRZ6p0aT8qV-ku+6BCQPt=3sZoa4vtxg@mail.gmail.com>
Message-ID: <CAA1VmgX+dA1yurOeoUicp4JbLrmmb1P6Ee+Og3Vs43cpmSahsQ@mail.gmail.com>

Hi,

I managed to achieve the results using only the pytest_report_teststatus. I
don't know if it is the best solution, but it now works as expected.

The code is available at https://github.com/felipecrp/pytest-pyspec

Regards,
Felipe

On Fri, Feb 24, 2023 at 11:17 AM Felipe Curty <felipecrp at gmail.com> wrote:

> Hello,
>
> I have been using pytest for a while and want to write a plugin to change
> its output.
>
> I would like to achieve the following output:
>
> From:
>
> test/test_sample.py .
>
> To
>
> test/test_sample.py
> A House
>  with two floors
>    has stairs                [Passed]
>
> I intend to use pytest_runtest_logreport in the setup phase to print the "A
> House with two floors has stairs". And to use pytest_report_teststatus to
> print the [Passed]. However, in `pytest_runtest_logreport`, I do not have
> access to TerminalReporter. Hence, apparently, the only alternative is to
> use the `print` command. Is there any problem using print instead of
> Terminal Reporter?
>
> I checked that some plugins already change the output, but they do this by
> changing the TerminalReporter class. For instance, pydev-it creates a
> subclass of the TerminalReporter (which is now a @final class and should
> not be extended), and pydev-spec patches some methods of the
> TerminalReporter.
>
> Regards,
> Felipe
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pytest-dev/attachments/20230224/22babef7/attachment.html>