[pytest-dev] py.test doesn't detect/warn on duplicate tests with parametrize

[holger krekel]

Hi Brianna,

was in vacation and then didn't have an immediate answer to offer ...

On Mon, Aug 19, 2013 at 16:51 +1000, Brianna Laugher wrote:
> Hi,
> 
> In our distributed tests we have been having intermittent failures on one
> particular parametrized test, with the failure always being on the same
> test case - failing in pytest_pyfunc_call in the look-up of a funcarg. This
> funcarg is used in a very large number of our tests (ie otherwise seems
> reliable) and has function scope.
> 
> I was just looking into it (traceback below) and I suddenly realised that
> the case was duplicated. Our test has 135 entries in parametrize so
> obviously someone (probably me!) just copy-pasted and accidentally left one
> line the same.
> 
> It seems to me it might be appropriate for py.test to warn about this, or
> even give a collection error? What do people think?

Maybe a warning would make sense.  There are also plans to internally
switch to manage parametrization by indexes into the parametrization
values (currently the ID is generated from the string representation of
values for numbers/strings/floats or some internal numbering for other
types).  With such a scheme, two identical parametrizations wouldn't
be considered wrong per se.

> Secondly does it seem like this might somehow be related to the
> intermittent test failure? It is baffling me.

It's hard to say, i am afraid.  I am still unhappy with the finalization
mechanics and it's quite possible something is going wrong there
related to your situation.

best,
holger