From yanghatespam at gmail.com  Thu Sep  5 02:03:04 2013
From: yanghatespam at gmail.com (Yang Zhang)
Date: Wed, 4 Sep 2013 17:03:04 -0700
Subject: [pytest-dev] How to exclude packages/directories from py.test?
Message-ID: <CAKxBDU_=XeMYMpuFg5zswJ_xfXgrbu6tDvfRnXuoGj-zqD6nrg@mail.gmail.com>

You can use -k to filter out tests following a certain name
pattern/predicate, but can you filter out tests under a certain
package or directory? Didn't spot any relevant flags in the
documentation but perhaps I'm missing something.  Thanks.

From yanghatespam at gmail.com  Thu Sep  5 11:58:32 2013
From: yanghatespam at gmail.com (Yang Zhang)
Date: Thu, 5 Sep 2013 02:58:32 -0700
Subject: [pytest-dev] How to exclude packages/directories from py.test?
In-Reply-To: <20130905095537.GA4012@merlinux.eu>
References: <CAKxBDU_=XeMYMpuFg5zswJ_xfXgrbu6tDvfRnXuoGj-zqD6nrg@mail.gmail.com>
 <20130905095537.GA4012@merlinux.eu>
Message-ID: <CAKxBDU8tmMCsw+=cJq3LZ5Lbq1B_jnMefhSZQS-v-j4xm=GbHg@mail.gmail.com>

Right, and we do use that already, but short of programmatically
generating a list of all the packages except those that I want to
exclude (given a fairly sprawling codebase), I was just wondering if
there was a cleaner approach?if not, consider this a feature request!

On Thu, Sep 5, 2013 at 2:55 AM, holger krekel <holger at merlinux.eu> wrote:
> On Wed, Sep 04, 2013 at 17:03 -0700, Yang Zhang wrote:
>> You can use -k to filter out tests following a certain name
>> pattern/predicate, but can you filter out tests under a certain
>> package or directory? Didn't spot any relevant flags in the
>> documentation but perhaps I'm missing something.  Thanks.
>
> py.test accepts files and directories::
>
>     py.test -k "..." path/to/dir
>
> will only consider tests from that dir.
>
> holger



-- 
Yang Zhang
http://yz.mit.edu/

From holger at merlinux.eu  Thu Sep  5 11:55:37 2013
From: holger at merlinux.eu (holger krekel)
Date: Thu, 5 Sep 2013 09:55:37 +0000
Subject: [pytest-dev] How to exclude packages/directories from py.test?
In-Reply-To: <CAKxBDU_=XeMYMpuFg5zswJ_xfXgrbu6tDvfRnXuoGj-zqD6nrg@mail.gmail.com>
References: <CAKxBDU_=XeMYMpuFg5zswJ_xfXgrbu6tDvfRnXuoGj-zqD6nrg@mail.gmail.com>
Message-ID: <20130905095537.GA4012@merlinux.eu>

On Wed, Sep 04, 2013 at 17:03 -0700, Yang Zhang wrote:
> You can use -k to filter out tests following a certain name
> pattern/predicate, but can you filter out tests under a certain
> package or directory? Didn't spot any relevant flags in the
> documentation but perhaps I'm missing something.  Thanks.

py.test accepts files and directories::

    py.test -k "..." path/to/dir

will only consider tests from that dir.

holger

From holger at merlinux.eu  Thu Sep  5 13:25:06 2013
From: holger at merlinux.eu (holger krekel)
Date: Thu, 5 Sep 2013 11:25:06 +0000
Subject: [pytest-dev] How to exclude packages/directories from py.test?
In-Reply-To: <CAKxBDU8tmMCsw+=cJq3LZ5Lbq1B_jnMefhSZQS-v-j4xm=GbHg@mail.gmail.com>
References: <CAKxBDU_=XeMYMpuFg5zswJ_xfXgrbu6tDvfRnXuoGj-zqD6nrg@mail.gmail.com>
 <20130905095537.GA4012@merlinux.eu>
 <CAKxBDU8tmMCsw+=cJq3LZ5Lbq1B_jnMefhSZQS-v-j4xm=GbHg@mail.gmail.com>
Message-ID: <20130905112506.GC4012@merlinux.eu>

On Thu, Sep 05, 2013 at 02:58 -0700, Yang Zhang wrote:
> Right, and we do use that already, but short of programmatically
> generating a list of all the packages except those that I want to
> exclude (given a fairly sprawling codebase), I was just wondering if
> there was a cleaner approach?if not, consider this a feature request!

FWIW the filename is also somewhat matchable, so "py.test -k 'not
somepackage'" should exclude all tests residing in a directory
"somepackage".  

holger


> On Thu, Sep 5, 2013 at 2:55 AM, holger krekel <holger at merlinux.eu> wrote:
> > On Wed, Sep 04, 2013 at 17:03 -0700, Yang Zhang wrote:
> >> You can use -k to filter out tests following a certain name
> >> pattern/predicate, but can you filter out tests under a certain
> >> package or directory? Didn't spot any relevant flags in the
> >> documentation but perhaps I'm missing something.  Thanks.
> >
> > py.test accepts files and directories::
> >
> >     py.test -k "..." path/to/dir
> >
> > will only consider tests from that dir.
> >
> > holger
> 
> 
> 
> -- 
> Yang Zhang
> http://yz.mit.edu/
> 

From holger at merlinux.eu  Fri Sep  6 12:27:44 2013
From: holger at merlinux.eu (holger krekel)
Date: Fri, 6 Sep 2013 10:27:44 +0000
Subject: [pytest-dev] support for --pdb on collect/internal errors
Message-ID: <20130906102744.GE4012@merlinux.eu>

Hi folks,

if you are hacking hooks or otherwise debugging pytest and plugins
you can now use "--pdb" when collect/internal errors appear.

For that, I have introduced a new experimental hook
"pytest_exception_interact" which could also be used 
by IDE integrators for their debugging views, e.g.:

    def pytest_exception_interact(call):
        excinfo = call.excinfo
        pydev.set_trace(excinfo.type, excinfo.value, excinfo.tb)

when this hook is called, stdout/stderr capturing will
be de-activated already.  It could also be used to spawn
up a http/socket server so that one could remotely connect 
to it for debugging sessions.

This is the commit:

https://bitbucket.org/hpk42/pytest/commits/1d3ebb313d68d4bcb3f1a2e575febf196689731b

cheers,

holger



From holger at merlinux.eu  Fri Sep  6 15:39:01 2013
From: holger at merlinux.eu (holger krekel)
Date: Fri, 6 Sep 2013 13:39:01 +0000
Subject: [pytest-dev] py.test doesn't detect/warn on duplicate tests
 with parametrize
In-Reply-To: <CAPjwCjv+_r2HcM7279M5FXSf+zaM4CHg-9YPmuFm+2B3ck0UGQ@mail.gmail.com>
References: <CAPjwCjv+_r2HcM7279M5FXSf+zaM4CHg-9YPmuFm+2B3ck0UGQ@mail.gmail.com>
Message-ID: <20130906133854.GK4012@merlinux.eu>

Hi Brianna,

was in vacation and then didn't have an immediate answer to offer ...

On Mon, Aug 19, 2013 at 16:51 +1000, Brianna Laugher wrote:
> Hi,
> 
> In our distributed tests we have been having intermittent failures on one
> particular parametrized test, with the failure always being on the same
> test case - failing in pytest_pyfunc_call in the look-up of a funcarg. This
> funcarg is used in a very large number of our tests (ie otherwise seems
> reliable) and has function scope.
> 
> I was just looking into it (traceback below) and I suddenly realised that
> the case was duplicated. Our test has 135 entries in parametrize so
> obviously someone (probably me!) just copy-pasted and accidentally left one
> line the same.
> 
> It seems to me it might be appropriate for py.test to warn about this, or
> even give a collection error? What do people think?

Maybe a warning would make sense.  There are also plans to internally
switch to manage parametrization by indexes into the parametrization
values (currently the ID is generated from the string representation of
values for numbers/strings/floats or some internal numbering for other
types).  With such a scheme, two identical parametrizations wouldn't
be considered wrong per se.

> Secondly does it seem like this might somehow be related to the
> intermittent test failure? It is baffling me.

It's hard to say, i am afraid.  I am still unhappy with the finalization
mechanics and it's quite possible something is going wrong there
related to your situation.

best,
holger

From holger at merlinux.eu  Fri Sep 13 12:33:47 2013
From: holger at merlinux.eu (holger krekel)
Date: Fri, 13 Sep 2013 10:33:47 +0000
Subject: [pytest-dev] monkeypatch.setattr versus replace
Message-ID: <20130913103347.GE4012@merlinux.eu>

Hi all,

this was discussed before on IRC, but here is an update and a question.
currently pytest-trunk implements a convenient new monkeypatch.replace method:

    # example usage
    monkeypatch.replace("os.path.abspath", lambda x: "/abc")

This effectively performs:

    import os.path
    monkeypatch.setattr(os.path, "abspath", lambda x: "/abc")

However, i just wanted to use it and couldn't remember the "replace"
name and don't find it very telling anymore.  I often do something
like this::

    monkeypatch.setattr("os.path.abspath", lambda x: "/abc")
   
which currently ends up as an error.  Would it be too awkward to
just make the latter work and forget about introducing a new "replace"
method name?  IMO it's rather intuitive to just make setattr accept
the new signature, although the docstring will need a bit of explaining.

cheers,
holger


From florian.schulze at gmx.net  Fri Sep 13 12:50:37 2013
From: florian.schulze at gmx.net (Florian Schulze)
Date: Fri, 13 Sep 2013 12:50:37 +0200
Subject: [pytest-dev] monkeypatch.setattr versus replace
In-Reply-To: <20130913103347.GE4012@merlinux.eu>
References: <20130913103347.GE4012@merlinux.eu>
Message-ID: <54064D42-E88A-427C-B368-D2BAE37F4451@gmx.net>


On 13.09.2013, at 12:33, holger krekel wrote:

> Hi all,
> 
> this was discussed before on IRC, but here is an update and a question.
> currently pytest-trunk implements a convenient new monkeypatch.replace method:
> 
>    # example usage
>    monkeypatch.replace("os.path.abspath", lambda x: "/abc")
> 
> This effectively performs:
> 
>    import os.path
>    monkeypatch.setattr(os.path, "abspath", lambda x: "/abc")
> 
> However, i just wanted to use it and couldn't remember the "replace"
> name and don't find it very telling anymore.  I often do something
> like this::
> 
>    monkeypatch.setattr("os.path.abspath", lambda x: "/abc")
> 
> which currently ends up as an error.  Would it be too awkward to
> just make the latter work and forget about introducing a new "replace"
> method name?  IMO it's rather intuitive to just make setattr accept
> the new signature, although the docstring will need a bit of explaining.

You can easily support both, so I would say go for it and just describe both ways in the docstring.

If you think the old way shouldn't be used, you can also add a deprecation warning to it.

Regards,
Florian Schulze


From holger at merlinux.eu  Fri Sep 13 12:54:42 2013
From: holger at merlinux.eu (holger krekel)
Date: Fri, 13 Sep 2013 10:54:42 +0000
Subject: [pytest-dev] monkeypatch.setattr versus replace
In-Reply-To: <54064D42-E88A-427C-B368-D2BAE37F4451@gmx.net>
References: <20130913103347.GE4012@merlinux.eu>
 <54064D42-E88A-427C-B368-D2BAE37F4451@gmx.net>
Message-ID: <20130913105442.GF4012@merlinux.eu>

On Fri, Sep 13, 2013 at 12:50 +0200, Florian Schulze wrote:
> On 13.09.2013, at 12:33, holger krekel wrote:
> 
> > Hi all,
> > 
> > this was discussed before on IRC, but here is an update and a question.
> > currently pytest-trunk implements a convenient new monkeypatch.replace method:
> > 
> >    # example usage
> >    monkeypatch.replace("os.path.abspath", lambda x: "/abc")
> > 
> > This effectively performs:
> > 
> >    import os.path
> >    monkeypatch.setattr(os.path, "abspath", lambda x: "/abc")
> > 
> > However, i just wanted to use it and couldn't remember the "replace"
> > name and don't find it very telling anymore.  I often do something
> > like this::
> > 
> >    monkeypatch.setattr("os.path.abspath", lambda x: "/abc")
> > 
> > which currently ends up as an error.  Would it be too awkward to
> > just make the latter work and forget about introducing a new "replace"
> > method name?  IMO it's rather intuitive to just make setattr accept
> > the new signature, although the docstring will need a bit of explaining.
> 
> You can easily support both, so I would say go for it and just describe both ways in the docstring.
> 
> If you think the old way shouldn't be used, you can also add a deprecation warning to it.

forgot to mention: monkeypatch.replace was never released, so we can just
remove it.  I take it you like the setattr-overload.

best,
holger

From anthon at mnt.org  Fri Sep 13 13:13:07 2013
From: anthon at mnt.org (Anthon van der Neut)
Date: Fri, 13 Sep 2013 13:13:07 +0200
Subject: [pytest-dev] monkeypatch.setattr versus replace
In-Reply-To: <20130913103347.GE4012@merlinux.eu>
References: <20130913103347.GE4012@merlinux.eu>
Message-ID: <5232F343.4070707@mnt.org>


On 2013-09-13 12:33, holger krekel wrote:
> Hi all,
>
> this was discussed before on IRC, but here is an update and a question.
> currently pytest-trunk implements a convenient new monkeypatch.replace 
> method:
>
>      # example usage
>      monkeypatch.replace("os.path.abspath", lambda x: "/abc")
>
> This effectively performs:
>
>      import os.path
>      monkeypatch.setattr(os.path, "abspath", lambda x: "/abc")
>
> However, i just wanted to use it and couldn't remember the "replace"
> name and don't find it very telling anymore.  I often do something
> like this::
>
>      monkeypatch.setattr("os.path.abspath", lambda x: "/abc")

As long as the old 3 parameter setattr is not going to be away, that 
seems fine.
Depending on the length of the string I would probably still use the 3 
parameter version to get the <TAB> completion on the first argument.
Unfortunately just writing

    monkeypatch.setattr(os.path.abspath, lambda x: "/abc")

in Python to get the same effect is out of the question AFAIK. Maybe I 
should try and make an/my editor smart enough to handle strings that are 
actually object-attribute-hierarchy references and do TAB completion on 
those.

>     which currently ends up as an error.  Would it be too awkward to
> just make the latter work and forget about introducing a new "replace"
> method name?  IMO it's rather intuitive to just make setattr accept
> the new signature, although the docstring will need a bit of explaining.
>
> cheers,
> holger
>
> _______________________________________________
> Pytest-dev mailing list
> Pytest-dev at python.org
> https://mail.python.org/mailman/listinfo/pytest-dev
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20130913/07955ef7/attachment.html>

From holger at merlinux.eu  Fri Sep 13 14:32:42 2013
From: holger at merlinux.eu (holger krekel)
Date: Fri, 13 Sep 2013 12:32:42 +0000
Subject: [pytest-dev] monkeypatch.setattr versus replace
In-Reply-To: <5232F30C.3060508@mnt.org>
References: <20130913103347.GE4012@merlinux.eu>
 <5232F30C.3060508@mnt.org>
Message-ID: <20130913123242.GG4012@merlinux.eu>

On Fri, Sep 13, 2013 at 13:12 +0200, Anthon van der Neut wrote:
> On 2013-09-13 12:33, holger krekel wrote:
> >Hi all,
> >
> >this was discussed before on IRC, but here is an update and a question.
> >currently pytest-trunk implements a convenient new monkeypatch.replace method:
> >
> >     # example usage
> >     monkeypatch.replace("os.path.abspath", lambda x: "/abc")
> >
> >This effectively performs:
> >
> >     import os.path
> >     monkeypatch.setattr(os.path, "abspath", lambda x: "/abc")
> >
> >However, i just wanted to use it and couldn't remember the "replace"
> >name and don't find it very telling anymore.  I often do something
> >like this::
> >
> >     monkeypatch.setattr("os.path.abspath", lambda x: "/abc")
> 
> As long as the old 3 parameter setattr is not going to be away, that
> seems fine.
> Depending on the length of the string I would probably still use the
> 3 parameter version to get the <TAB> completion on the first
> argument.
> Unfortunately just writing
> 
>    monkeypatch.setattr(os.path.abspath, lambda x: "/abc")

It's actually possible to make this work in many common cases,
but is bordering on the magic side.

holger


From flub at devork.be  Mon Sep 16 14:11:54 2013
From: flub at devork.be (Floris Bruynooghe)
Date: Mon, 16 Sep 2013 13:11:54 +0100
Subject: [pytest-dev] monkeypatch.setattr versus replace
In-Reply-To: <20130913123242.GG4012@merlinux.eu>
References: <20130913103347.GE4012@merlinux.eu> <5232F30C.3060508@mnt.org>
 <20130913123242.GG4012@merlinux.eu>
Message-ID: <CAAWk_DywdvBk2Aj6KkWaoj8RGnpnQTqRX6DZXj_7Ebe4+=u07A@mail.gmail.com>

[resending to list - apologies]

On 13 September 2013 13:32, holger krekel <holger at merlinux.eu> wrote:
>>    monkeypatch.setattr(os.path.abspath, lambda x: "/abc")
>
> It's actually possible to make this work in many common cases,
> but is bordering on the magic side.

This would actually be my favourite and is what I usually type when
I'm not thinking.  However there is nothing wrong with the
two-argument version where the first is a string either, so I'm +1 on
that.  Accepting this form is then a separate issue to consider.


Regards,
Floris

From anto.cuni at gmail.com  Tue Sep 17 10:32:18 2013
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 17 Sep 2013 10:32:18 +0200
Subject: [pytest-dev] monkeypatch.setattr versus replace
In-Reply-To: <20130913103347.GE4012@merlinux.eu>
References: <20130913103347.GE4012@merlinux.eu>
Message-ID: <52381392.2070703@gmail.com>

Hi,

On 13/09/13 12:33, holger krekel wrote:
>      monkeypatch.setattr("os.path.abspath", lambda x: "/abc")

speaking about possible APIs, what about the following?

monkeypatch("os.path.abspath = foo")

I find it much more readable than the two or three argument versions, and it's 
in a way similar to what we have for py.test.raises("...").
The drawback is that the implementation is probably a bit trickier than what 
it is now.

ciao,
Anto

From holger at merlinux.eu  Tue Sep 17 12:13:34 2013
From: holger at merlinux.eu (holger krekel)
Date: Tue, 17 Sep 2013 10:13:34 +0000
Subject: [pytest-dev] monkeypatch / mock / higher level helpers
In-Reply-To: <52381392.2070703@gmail.com>
References: <20130913103347.GE4012@merlinux.eu>
 <52381392.2070703@gmail.com>
Message-ID: <20130917101334.GK4012@merlinux.eu>

On Tue, Sep 17, 2013 at 10:32 +0200, Antonio Cuni wrote:
> On 13/09/13 12:33, holger krekel wrote:
> >     monkeypatch.setattr("os.path.abspath", lambda x: "/abc")
> 
> speaking about possible APIs, what about the following?
> 
> monkeypatch("os.path.abspath = foo")
> 
> I find it much more readable than the two or three argument
> versions, and it's in a way similar to what we have for
> py.test.raises("...").
> The drawback is that the implementation is probably a bit trickier
> than what it is now.

Not sure it would even be that useful TBH and would indeed be trickier
to implement, i guess.  Maybe it's better to rather think about
higher level helpers than argue much about the setattr() convenience.

Apart from the proposed setattr shortcut, i am also thinking of
direct mock (the lib from Michael Foord) support:

    mock = monkeypatch.setmock("os.path.abspath")
    call_something_with_triggers_the_mock()
    assert mock.called_once_with(...)

With >=py3.3 this could just work, on older Pythons ``setmock``
would skip the test if "mock" is not importable.

However, i dislike parts of the mock API design, in particular that
the "mock" object is send to applications but also provides assertions
and mock-specific data attributes.  If your app has attributes or
methods like "method_calls", "called", "call_args" etc.  then you can
not mock it.  Instead mock's API should rather work like this:

    m = mock.Mock()
    ... pass m somewhere ... (having virtually no special attributes)
    assert mock.assert_once_called_with(m, ...)

This way "mock.*" and assertions would become a "meta" api, strictly 
separated from the mock objects that are sent into an application,
avoid any conflicts.  Practicality might beat purity, though. mock
is already widely used ...
       
cheers,
holger

From anto.cuni at gmail.com  Tue Sep 17 14:46:24 2013
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 17 Sep 2013 14:46:24 +0200
Subject: [pytest-dev] monkeypatch / mock / higher level helpers
In-Reply-To: <20130917101334.GK4012@merlinux.eu>
References: <20130913103347.GE4012@merlinux.eu> <52381392.2070703@gmail.com>
 <20130917101334.GK4012@merlinux.eu>
Message-ID: <52384F20.2020001@gmail.com>

On 17/09/13 12:13, holger krekel wrote:
> Not sure it would even be that useful TBH and would indeed be trickier
> to implement, i guess.  Maybe it's better to rather think about
> higher level helpers than argue much about the setattr() convenience.

I think that designing a good API is not only about convenience, but it has a 
high impact on how easy it is to learn to use the tool.

I just though of another variation on the theme:

monkeypatch(os.path).abspath = foobar # OR
monkey(os.path).abspath = foobar

I think this is the easiest to remember, to understand and it's also probably 
very easy to implement.


> Apart from the proposed setattr shortcut, i am also thinking of
> direct mock (the lib from Michael Foord) support:
>
>      mock = monkeypatch.setmock("os.path.abspath")

not sure it's a good idea. I don't think it is much simpler than this:

     mock = monekypatch.setattr("os.path.abspath", Mock())

apart that in the latter case you must import Mock explicitly, which is a good 
thing IMHO.



From holger at merlinux.eu  Tue Sep 17 15:07:48 2013
From: holger at merlinux.eu (holger krekel)
Date: Tue, 17 Sep 2013 13:07:48 +0000
Subject: [pytest-dev] monkeypatch / mock / higher level helpers
In-Reply-To: <52384F20.2020001@gmail.com>
References: <20130913103347.GE4012@merlinux.eu> <52381392.2070703@gmail.com>
 <20130917101334.GK4012@merlinux.eu> <52384F20.2020001@gmail.com>
Message-ID: <20130917130748.GL4012@merlinux.eu>

On Tue, Sep 17, 2013 at 14:46 +0200, Antonio Cuni wrote:
> On 17/09/13 12:13, holger krekel wrote:
> >Not sure it would even be that useful TBH and would indeed be trickier
> >to implement, i guess.  Maybe it's better to rather think about
> >higher level helpers than argue much about the setattr() convenience.
> 
> I think that designing a good API is not only about convenience, but
> it has a high impact on how easy it is to learn to use the tool.
> 
> I just though of another variation on the theme:
> 
> monkeypatch(os.path).abspath = foobar # OR
> monkey(os.path).abspath = foobar

> I think this is the easiest to remember, to understand and it's also
> probably very easy to implement.

not sure it's easy to remember given that several different somewhat
popular approaches all don't look like this.  But it's a cute trick 
that indeed wouldn't be hard to implement :)

> >Apart from the proposed setattr shortcut, i am also thinking of
> >direct mock (the lib from Michael Foord) support:
> >
> >     mock = monkeypatch.setmock("os.path.abspath")
> 
> not sure it's a good idea. I don't think it is much simpler than this:
> 
>     mock = monekypatch.setattr("os.path.abspath", Mock())
> 
> apart that in the latter case you must import Mock explicitly, which
> is a good thing IMHO.

Not sure i agree with "importing is a good thing" :) For fixtures people
have come to appreciate, i think, the de-coupling of using values and
providing them.  Mocking is tightly connected with testing so having
a builtin support fixture is reasonable IIRC.

Also, with monkeypatch.setattr(..., Mock()) you cannot easily do "autospec"
whereas with monkeypatch.setmock(...) you could (And disable with
autospec=False).  Or maybe a new "mock" and "mock.patch(...)" would
be even easier for people knowing mock's API -- except that "mock"
would be a fixture provided by pytest or a plugin, and mock.patch
would be an imperative call.

best,
holger

From vladimir at keleshev.com  Sat Sep 28 22:29:59 2013
From: vladimir at keleshev.com (Vladimir Keleshev)
Date: Sat, 28 Sep 2013 22:29:59 +0200
Subject: [pytest-dev] Case for @fixture to support yield-style fixtures
	directly
Message-ID: <423861380400199@web15j.yandex.ru>

I would like to make a case for making yield-style fixtures
to be supported directly by the @fixture decorator.

But before that I want to remind some of you on what I am
talking about.

Yield-sytle fixtures are planned to be added in pytest 2.4,
but their exact API is under discussion.

Let me remind you that yield-style fixture looks like:

    @fixture
    def empty_db():
        db = DB()           # Setup code.
        yield db
        db.clear().close()  # Teardown code.

Now contrast with pytest 2.3 fixtures:

    @fixture
    def empty_db(request):
        db = DB()
        request.addfinalizer(lambda: db.clear().close())
        return db

Both create an instance of a hypothetical database,
return it as a fixture, then perform cleanup. For a similar
real-life example take a look here:
http://github.com/halst/urlbeat/blob/master/test_urlbeat.py

Current docs draft (http://pytest.org/dev/yieldfixture.html)
states some advantages and disadvantages which I want to
comment on.

1.1 Streamlined control flow.

Callbacks are known to (visually) break the order of
control flow, which contributes to their poor readability.
At the same time, having tear-down code after `yield` might
seem unusuall at first, but I think that such fixtures are
so readable to the point of being called beautiful.  You
can read them from top to bottom: first, set-up code, then
the object is yielded, then the tear-down follows.

1.2 No need for `request` argument.

In my experience with pytest 2.3 and before, the `request`
argument of fixtures is used for defining finalizers in
about 90% of the cases.  Other uses are relatively rare.
Yield-sytle fixtures allow to simplify this by allowing 
to drop that argument.

1.3. Simple fixtures that use context managers.

This is important; yield-style fixtures allow to use
context managers (think `with` statement) inside fixtures.
For example, if a `DB` class is designed to be a context
manager, you would write the following yield-style fixture:

    @fixture
    def emtpy_db():
        with DB() as db:
            yield db

However, with current pytest 2.3 fixtures that would 
approximately be:

    @fixture
    def empty_db(request):
        db = DB.__enter__()
        request.addfinalizer(
            lambda: db.__exit__(None, None, None))
        return db

What I want to stress is that context managers were
specifically introduced for this purpose: handle resources
that need some set-up and tear-down.  Almost all such
resources now support them.  That is one of the reasons,
I think, that yield-style fixtures are such a huge
improvement.

* * *

To sum up the advantages: you can argue about readability,
but you can't argue that support for context-managers is
an extremely important use-case.  Both fixtures and context
managers are made primarily to set-up and tear-down
resources. Increasing numbers of API support context
managers, and I'm sure that most of existing fixtures
(that use `.close()` and `.clear()` and similar methods)
could be rewritten more elegantly (and more uniformly) to
use yield-style fixtures with context managers.

* * *

Now, to comment on disadvantages:

2.1. Existing fixtures might use `yield` for ad-hoc
parametrization.

That means that yield-sytle fixtures might break existing
code that uses generators for parametrization like this:

    @fixture
    def all_backends(request):
        for Backend in [MemoryBackend,
                        BinaryBackend,
                        CSVBackend]:
            backend = Backend()
            yield backend
            request.addfinalizer(backend.clear)

    def test_backends(all_backends):
        for backend in all_backends:
            assert backend.put('key', 42).get('key') == 42

For one, I have never seen such code, and for two, the same
could be achieved more elegantly with parametrization API:

    @fixture(params=[MemroyBackend,
                     BinaryBackend,
                     CSVBackend])
    def backend(request):
        backend = request.param()
        request.addfinilizer(backend.clear)
        return backend

    def test_backends(backend):
        assert backend.put('key', 42).get('key') == 42

Plus, as of pytest 2.4.0.dev11, the user of such ad-hoc
generator fixture will see a very clear error message:

    fixture function backends has more than one `yield`

Which could elaborate even more if necessary:

    Fixture function `backends` has more than one `yield`.
    Since pytest 2.4 fixtures with `yield` statement are
    treated differently: <link>.  You might want to use
    parametrization API instead: <link>.

2.2 Yield-style fixtures introduce more that one way to do
fixtures (together with current return-style fixtures)

Yield-style fixtures present us with a superset of
functionality of what return-style fixtures can achieve.
I would even like to say that except for backwards-compatibility
and user habit concerns, there is no need for return-style
fixtures at all, if yield-style is available.  So I would
even argue that yield-style fixtures should take the main
place in fixture documentation, and return-style fixtures
mentioned for backwards compatibility (as funcargs are).

* * *

To sum up, the only real disadvantage of yield-style
fixtures is that they can break a single hypothetical
use-case, which is either rare or nonexistent, and for 
which a clear error-message is provided.

On the other hand, they allow to use context managers for
finalization, which were specifically designed to hanldle
this use-case elegantly.

I'm pretty sure that 100% of fixtures that use
`request.addfinalizer` will benefit from yield-style
fixtures, and that most of them will be able to use
context managers for finalization.

Just look at all the evolution steps.

3.1 Past:

    def pytest_funcarg__db(request):
        db = DB()
        request.addfinalizer(db.clear)
        return db

3.2 Present:

    @fixture
    def db(request):
        db = DB()
        request.addfinalizer(db.clear)
        return db

3.3 Future:

    @fixture
    def db():
        db = DB()
        yield db
        db.clear()

Or even better:

    @fixture
    def db():
        with DB() as db:
            yield db

So, do you agree that this is the future we want to be in?

* * *

There are a certain challenges that we need to solve then.

4.1 Backwards-compatibility (which I already mentioned).

4.2 There is a high chance (in my experience) of mistyping
`return` for `yield` like:

    @fixture
    def db():
        db = DB()
        return db
        db.clear()

Even worse, this fixture could still appear to work, until
some later time, when they system runs out of file
descriptors, for example.

Here are the 3 solutions discussed:

5.1 Extra argument to `fixture` to designate yield-style:

    @fixture(yield_context=True)
    def db():
        db = DB()
        yield db
        db.clear()

5.2 Different fixture decorator to designate yield-sytle:

    @yield_fixture
    def db():
        db = DB()
        yield db
        db.clear()

5.3 A configuration option, say, in pytest.ini that will
switch bettween return-style and yield-style.

What I would like to propose is:

6.1 Keep single @fixture decorator.

6.2 Give user a clear error-message in case fixture yields
more than one time (done).

6.3 Using simple static-analysis, warn user when there
are statements after `return`, to get rid of return/yield
typos.

4. Document yield-style as the preferable way of
finalization.

All of that keeps the API simple (no new decorators,
arguments, configuration options), warns user in case of
every known caveat, and finally, provides us the beautiful
yield-style fixtures by default.

?Vladimir Keleshev

From holger at merlinux.eu  Sun Sep 29 08:58:08 2013
From: holger at merlinux.eu (holger krekel)
Date: Sun, 29 Sep 2013 06:58:08 +0000
Subject: [pytest-dev] Case for @fixture to support yield-style fixtures
 directly
In-Reply-To: <423861380400199@web15j.yandex.ru>
References: <423861380400199@web15j.yandex.ru>
Message-ID: <20130929065807.GN14010@merlinux.eu>

Hey Vladimir,

thanks for your considerations (relating to
http://pytest.org/dev/yieldfixture.html )

I am going to write a condensed reply to the few points where i disagree.

TLDR, i suggest to go for ``yield_fixture`` for pytest-2.4.

On Sat, Sep 28, 2013 at 22:29 +0200, Vladimir Keleshev wrote:
> Now, to comment on disadvantages:
> Yield-style fixtures present us with a superset of
> functionality of what return-style fixtures can achieve.
> I would even like to say that except for backwards-compatibility
> and user habit concerns, there is no need for return-style
> fixtures at all, if yield-style is available.  So I would
> even argue that yield-style fixtures should take the main
> place in fixture documentation, and return-style fixtures
> mentioned for backwards compatibility (as funcargs are).

We can discuss this for 2.5 but not for 2.4.  It's too
massive a change at this point.  Also, from giving
courses to non-expert Pythonistas i know that "yield"
(as much as inheritance and other concepts) are not easy
concepts if if they are in computer science.

> 4.2 There is a high chance (in my experience) of mistyping
> `return` for `yield` like:
> 
>     @fixture
>     def db():
>         db = DB()
>         return db
>         db.clear()
> 
> Even worse, this fixture could still appear to work, until
> some later time, when they system runs out of file
> descriptors, for example.

I think this error is highly unlikely because "return" for
almost everybody marks the end of the function.

So for pytest-2.4 we are left with a choice between a new
"yield_fixture" decorator or passing a ``yieldctx=True``
to the existing one.  To make it easier for people to use
yield fixtures everywhere a decorator might be preferable.
You could write:

    from pytest import yield_fixture as fixture

and use yield style everywhere.  Not using yield in a 
yield_fixture then would get you an error.  Using
yield in a @pytest.fixture would work as before.

Originally i didn't want to go with a separate decorator
because it duplicates API and docstrings.  But maybe this can
be mediated by just saying "same as pytest.fixture but
use 'yield' instead of 'return" for providing a fixture instance, 
see http:/... for details."

Sounds like a plan?

holger

From florian.schulze at gmx.net  Sun Sep 29 11:41:58 2013
From: florian.schulze at gmx.net (Florian Schulze)
Date: Sun, 29 Sep 2013 11:41:58 +0200
Subject: [pytest-dev] Case for @fixture to support yield-style fixtures
	directly
In-Reply-To: <20130929065807.GN14010@merlinux.eu>
References: <423861380400199@web15j.yandex.ru>
 <20130929065807.GN14010@merlinux.eu>
Message-ID: <8BBD6CE1-A495-4317-803A-37324365AD83@gmx.net>


On 29.09.2013, at 08:58, holger krekel wrote:

> TLDR, i suggest to go for ``yield_fixture`` for pytest-2.4.

+1

From holger at merlinux.eu  Sun Sep 29 13:15:41 2013
From: holger at merlinux.eu (holger krekel)
Date: Sun, 29 Sep 2013 11:15:41 +0000
Subject: [pytest-dev] Case for @fixture to support yield-style fixtures
 directly
In-Reply-To: <8BBD6CE1-A495-4317-803A-37324365AD83@gmx.net>
References: <423861380400199@web15j.yandex.ru>
 <20130929065807.GN14010@merlinux.eu>
 <8BBD6CE1-A495-4317-803A-37324365AD83@gmx.net>
Message-ID: <20130929111540.GO14010@merlinux.eu>

On Sun, Sep 29, 2013 at 11:41 +0200, Florian Schulze wrote:
> On 29.09.2013, at 08:58, holger krekel wrote:
> 
> > TLDR, i suggest to go for ``yield_fixture`` for pytest-2.4.
> 
> +1

coming back from the children playground i am even thinking
about "yfixture" to keep things short :)


From florian.schulze at gmx.net  Sun Sep 29 14:46:46 2013
From: florian.schulze at gmx.net (Florian Schulze)
Date: Sun, 29 Sep 2013 14:46:46 +0200
Subject: [pytest-dev] Case for @fixture to support yield-style fixtures
	directly
In-Reply-To: <20130929111540.GO14010@merlinux.eu>
References: <423861380400199@web15j.yandex.ru>
 <20130929065807.GN14010@merlinux.eu>
 <8BBD6CE1-A495-4317-803A-37324365AD83@gmx.net>
 <20130929111540.GO14010@merlinux.eu>
Message-ID: <DF4A27DB-C283-430B-8D20-064E5B70E5B7@gmx.net>


On 29.09.2013, at 13:15, holger krekel wrote:

> On Sun, Sep 29, 2013 at 11:41 +0200, Florian Schulze wrote:
>> On 29.09.2013, at 08:58, holger krekel wrote:
>> 
>>> TLDR, i suggest to go for ``yield_fixture`` for pytest-2.4.
>> 
>> +1
> 
> coming back from the children playground i am even thinking
> about "yfixture" to keep things short :)

Nah, people can do that at import time if they want.

Regards,
Florian Schulze


From holger at merlinux.eu  Mon Sep 30 16:06:33 2013
From: holger at merlinux.eu (holger krekel)
Date: Mon, 30 Sep 2013 14:06:33 +0000
Subject: [pytest-dev] please test pytest-2.4.0 release candidate
Message-ID: <20130930140633.GQ14010@merlinux.eu>

Hi everybody,

could you help to verify that pytest-2.4 will continue to work for your
test suite and/or plugins?  Please report back any issues here or on
#pylib freenode.  Release is scheduled tomorrow (tuesday) noon (GMT) so
please hurry :)

The repo is now pretty much in shape for a release and
you can install a packaged version like this::

    pip install -i http://devpi.net/hpk/dev/+simple/ -U pytest

this should give you pytest-2.4.dev13 (see "--version" option).

Below is the long changelog in case you are curious.  

Latest two changes:

- the decorator yield_fixture is now there.  See
  http://pytest.org/dev/yieldfixture.html

- i did a refactoring of how pytest startup is performed, fixing
  issue358 and allowing pytest-django and other plugins to 
  perform actions before the first conftest file is loaded.
  Andreas Pelme already adapted pytest-django to utilize
  the improvements (mainly you can then use ``--ds=...`` setting
  and have your conftest.py files import django) and it seems to fix
  issue359.

Apart from some known incompatibilities (see changelog below)
pytest-2.4 aims to be a drop-in replacement for 2.3.5.

cheers,
holger


Changes between 2.3.5 and 2.4.DEV
-----------------------------------

known incompatibilities:

- if calling --genscript from python2.7 or above, you only get a
  standalone script which works on python2.7 or above.  Use Python2.6
  to also get a python2.5 compatible version.

- all xunit-style teardown methods (nose-style, pytest-style,
  unittest-style) will not be called if the corresponding setup method failed,
  see issue322 below.

- the pytest_plugin_unregister hook wasn't ever properly called
  and there is no known implementation of the hook - so it got removed.

- pytest.fixture-decorated functions cannot be generators (i.e. use
  yield) anymore.  This change might be reversed in 2.4.1 if it causes
  unforeseen real-life issues.  However, you can always write and return
  an inner function/generator and change the fixture consumer to iterate
  over the returned generator.  This change was done in lieu of the new
  ``pytest.yield_fixture`` decorator, see below.

new features:

- experimentally introduce a new ``pytest.yield_fixture`` decorator
  which accepts exactly the same parameters as pytest.fixture but
  mandates a ``yield`` statement instead of a ``return statement`` from
  fixture functions.  This allows direct integration with "with-style"
  context managers in fixture functions and generally avoids registering
  of finalization callbacks in favour of treating the "after-yield" as
  teardown code.  Thanks Andreas Pelme, Vladimir Keleshev, Floris
  Bruynooghe, Ronny Pfannschmidt and many others for discussions.

- allow boolean expression directly with skipif/xfail
  if a "reason" is also specified.  Rework skipping documentation
  to recommend "condition as booleans" because it prevents surprises
  when importing markers between modules.  Specifying conditions
  as strings will remain fully supported.

- reporting: color the last line red or green depending if
  failures/errors occured or everything passed.  thanks Christian
  Theunert.

- make "import pdb ; pdb.set_trace()" work natively wrt capturing (no
  "-s" needed anymore), making ``pytest.set_trace()`` a mere shortcut.

- fix issue181: --pdb now also works on collect errors (and 
  on internal errors) .  This was implemented by a slight internal 
  refactoring and the introduction of a new hook 
  ``pytest_exception_interact`` hook (see next item).

- fix issue341: introduce new experimental hook for IDEs/terminals to 
  intercept debugging: ``pytest_exception_interact(node, call, report)``.

- new monkeypatch.setattr() variant to provide a shorter
  invocation for patching out classes/functions from modules:

     monkeypatch.setattr("requests.get", myfunc)

  will replace the "get" function of the "requests" module with ``myfunc``.

- fix issue322: tearDownClass is not run if setUpClass failed. Thanks
  Mathieu Agopian for the initial fix.  Also make all of pytest/nose
  finalizer mimick the same generic behaviour: if a setupX exists and
  fails, don't run teardownX.  This internally introduces a new method
  "node.addfinalizer()" helper which can only be called during the setup
  phase of a node.

- simplify pytest.mark.parametrize() signature: allow to pass a
  CSV-separated string to specify argnames.  For example: 
  ``pytest.mark.parametrize("input,expected",  [(1,2), (2,3)])``
  works as well as the previous:
  ``pytest.mark.parametrize(("input", "expected"), ...)``.

- add support for setUpModule/tearDownModule detection, thanks Brian Okken.

- integrate tab-completion on options through use of "argcomplete".
  Thanks Anthon van der Neut for the PR.

- change option names to be hyphen-separated long options but keep the
  old spelling backward compatible.  py.test -h will only show the
  hyphenated version, for example "--collect-only" but "--collectonly"
  will remain valid as well (for backward-compat reasons).  Many thanks to
  Anthon van der Neut for the implementation and to Hynek Schlawack for
  pushing us.

- fix issue 308 - allow to mark/xfail/skip individual parameter sets
  when parametrizing.  Thanks Brianna Laugher.

- call new experimental pytest_load_initial_conftests hook to allow
  3rd party plugins to do something before a conftest is loaded.

Bug fixes:

- fix issue358 - capturing options are now parsed more properly 
  by using a new parser.parse_known_args method.

- pytest now uses argparse instead of optparse (thanks Anthon) which 
  means that "argparse" is added as a dependency if installing into python2.6
  environments or below.

- fix issue333: fix a case of bad unittest/pytest hook interaction.

- PR27: correctly handle nose.SkipTest during collection.  Thanks
  Antonio Cuni, Ronny Pfannschmidt.

- fix issue355: junitxml puts name="pytest" attribute to testsuite tag.

- fix issue336: autouse fixture in plugins should work again.

- fix issue279: improve object comparisons on assertion failure
  for standard datatypes and recognise collections.abc.  Thanks to
  Brianna Laugher and Mathieu Agopian.

- fix issue317: assertion rewriter support for the is_package method

- fix issue335: document py.code.ExceptionInfo() object returned
  from pytest.raises(), thanks Mathieu Agopian.

- remove implicit distribute_setup support from setup.py.

- fix issue305: ignore any problems when writing pyc files.

- SO-17664702: call fixture finalizers even if the fixture function
  partially failed (finalizers would not always be called before)

- fix issue320 - fix class scope for fixtures when mixed with
  module-level functions.  Thanks Anatloy Bubenkoff.

- you can specify "-q" or "-qq" to get different levels of "quieter"
  reporting (thanks Katarzyna Jachim)

- fix issue300 - Fix order of conftest loading when starting py.test
  in a subdirectory.

- fix issue323 - sorting of many module-scoped arg parametrizations

- make sessionfinish hooks execute with the same cwd-context as at
  session start (helps fix plugin behaviour which write output files 
  with relative path such as pytest-cov)

- fix issue316 - properly reference collection hooks in docs

- fix issue 306 - cleanup of -k/-m options to only match markers/test
  names/keywords respectively.  Thanks Wouter van Ackooy.

- improved doctest counting for doctests in python modules -- 
  files without any doctest items will not show up anymore
  and doctest examples are counted as separate test items.
  thanks Danilo Bellini.

- fix issue245 by depending on the released py-1.4.14
  which fixes py.io.dupfile to work with files with no
  mode. Thanks Jason R. Coombs.

- fix junitxml generation when test output contains control characters,
  addressing issue267, thanks Jaap Broekhuizen 

- fix issue338: honor --tb style for setup/teardown errors as well.  Thanks Maho.

- fix issue307 - use yaml.safe_load in example, thanks Mark Eichin.

- better parametrize error messages, thanks Brianna Laugher

- pytest_terminal_summary(terminalreporter) hooks can now use
  ".section(title)" and ".line(msg)" methods to print extra 
  information at the end of a test run.