[pytest-dev] [pytest-rerunfailures] pytest-rerunfailures not using fixtures on reruns (#10)

[Leah Klearman]

Hey Holger,

Thank you for the timely patch. I haven't seen Bob online today, but
hopefully he'll be able to test it very soon.

A word of warning: your calling of runtestprotocol() is not quite right
> and might lead to problems.  "nextitem" should really be the item that
> is going to be run next.  So if you re-run three times the first two
> invocations should have nextitem=item.


I agree that that would be ideal, but it only re-runs the test if it fails,
so I
would have to know the future in order to know what value to send nextitem=.
Since hopefully most tests pass the first try, sending nextitem instead of
item seems to be the better answer.

Thanks!
-Leah


On Mon, Apr 22, 2013 at 1:27 AM, holger krekel <holger at merlinux.eu> wrote:

> Hi Leah, Bob,
>
> On Fri, Apr 19, 2013 at 20:55 -0700, Leah Klearman wrote:
> > Hi Holger and other py.test mavens,
> >
> > Bob has reported a problem with my py.test plugin pytest-rerunfailures
> [1]
> > not re-running the setup before rerunning the test.
> >
> > Looking at my code, it has pytest_runtest_protocol() [2] looping
> > on _pytest.runner.runtestprotocol() [3], which in turn runs the setup,
> the
> > test, and the teardown.
> >
> > [1] https://github.com/klrmn/pytest-rerunfailures
> > [2]
> >
> https://github.com/klrmn/pytest-rerunfailures/blob/master/rerunfailures/plugin.py#L46
> > [3]
> >
> https://bitbucket.org/hpk42/pytest/src/fdc28ac2029f1c0be1dac4991c2f1b014c39a03f/_pytest/runner.py?at=default#cl-65
> >
> > I haven't taken the hours needed to get my head fully into py.test plugin
> > development mode, but I'm not sure I can implement a fix at my layer.
> >
> >
> > I'm hoping someone here will have some insight.
>
> It's a bit intricate.  A function item keeps around some fixture state
> and was so far not intended to be run multiple times.  I went ahead and
> tried to improve the behaviour to better allow re-running.   Please try
> with
>
>     pip install -i http://pypi.testrun.org -U pytest
>
> which should give you pytest-2.3.5.dev16 at least.  This is bound to be
> released soon so quick feedback is welcome.  If you still have problems
> please try to send a minimal test file which shows undesired behaviour.
>
> A word of warning: your calling of runtestprotocol() is not quite right
> and might lead to problems.  "nextitem" should really be the item that
> is going to be run next.  So if you re-run three times the first two
> invocations should have nextitem=item.
>
> best,
> holger
>
>
> > Thanks,
> > -Leah
> >
> >
> > On Sun, Apr 14, 2013 at 6:29 PM, Bob Silverberg <
> notifications at github.com>wrote:
> >
> > > I just verified this behaviour myself with a simple test [1]. I see it
> > > with both funcargs and fixtures, but I'm not sure if it has to do with
> the
> > > plugin, or the way py.test works. It does inject the value into the
> test
> > > method, but it doesn't rerun the fixture, so it seems like it is
> caching
> > > the first run of the fixture and using that on subsequent runs.
> > >
> > > I'm not sure if this is something that the plugin can have any effect
> on,
> > > or if it's just the way fixtures work. It is specified for this fixture
> > > that it is scope='function', and perhaps py.test makes that happen by
> > > checking the function name, which is, of course, the same for each
> run. I
> > > did try removing the scope argument from the fixture but that had no
> > > effect.
> > >
> > > Do you have any thoughts about this, @klrmn <https://github.com/klrmn
> >?
> > >
> > > [1] https://gist.github.com/bobsilverberg/5385035
> > >
> > > —
> > > Reply to this email directly or view it on GitHub<
> https://github.com/klrmn/pytest-rerunfailures/issues/10#issuecomment-16363644
> >
> > > .
> > >
>
> > _______________________________________________
> > Pytest-dev mailing list
> > Pytest-dev at python.org
> > http://mail.python.org/mailman/listinfo/pytest-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20130422/c544bcf0/attachment.html>