[Pytest-commit] Issue #423: Consecutive pytest.mark.skipif inside pytest.mark.parameterize (hpk42/pytest)

[Ldiary Translations]

New issue 423: Consecutive pytest.mark.skipif inside pytest.mark.parameterize
https://bitbucket.org/hpk42/pytest/issue/423/consecutive-pytestmarkskipif-inside

Ldiary Translations:

Given the test module below which was derived from the documentation example:
http://pytest.org/latest/parametrize.html#pytest-mark-parametrize-parametrizing-test-functions


```
#!python

import pytest

flag = []


@pytest.mark.default
@pytest.mark.parametrize("my_input,expected", [
    ("3+5", 8),
    ("2+4", 6),
    ("6*9", 42),
])
def test_default_example(my_input, expected):
    assert eval(my_input) == expected


@pytest.mark.skipsecond
@pytest.mark.parametrize("my_input,expected", [
    ("3+5", "Failed"),
    pytest.mark.skipif("flag == []", ("2+4", 6)),
    ("6*9", 42),
])
def test_skip_second(my_input, expected):
    print("\nBefore clearing flag: ", flag)
    while len(flag) > 0:
        flag.pop()
    print("After clearing flag: ", flag)
    assert eval(my_input) == expected
    flag.append("Previous Test Passed")


@pytest.mark.skipthird
@pytest.mark.parametrize("my_input,expected", [
    ("3+5", 8),
     ("2+4", "Failed"),
    pytest.mark.skipif("flag == []", ("6*9", 42)),
])
def test_skip_third(my_input, expected):
    print("\nBefore clearing flag: ", flag)
    while len(flag) > 0:
        flag.pop()
    print("After clearing flag: ", flag)
    assert eval(my_input) == expected
    flag.append("Previous Test Passed")


@pytest.mark.skipsecondthird
@pytest.mark.parametrize("my_input,expected", [
    ("3+5", 8),
    pytest.mark.skipif("flag == []", ("2+4", "Failed")),
    pytest.mark.skipif("flag == []", ("6*9", 42)),
])
def test_skip_second_third(my_input, expected):
    print("\nBefore clearing flag: ", flag)
    while len(flag) > 0:
        flag.pop()
    print("After clearing flag: ", flag)
    assert eval(my_input) == expected
    flag.append("Previous Test Passed")
```

The test marks "default", "skipsecond" and "skipthird" works as expected, but the test mark "skipsecondthird" has strange behaviour.

We can't understand why when the "skipsecondthird" test mark is executed like below, the last test was still executed three times. We are expecting that the first test execution will be PASSED, the second test execution will FAILED, and the third execution will be SKIPPED; but it wasn't.

Are consecutive skipif inside parameterize supported on the first place?
Or is this a bug?

$ py.test -svm skipsecondthird 

========== test session starts =============================

platform linux -- Python 3.3.3 -- pytest-2.5.0 -- /home/ldiary/py3env/bin/python3

SecondSkipInParameterized.py:47: test_skip_second_third[3+5-8] 

Before clearing flag:  []

After clearing flag:  []

PASSED

SecondSkipInParameterized.py:47: test_skip_second_third[2+4-Failed] 

Before clearing flag:  ['Previous Test Passed']

After clearing flag:  []

FAILED

SecondSkipInParameterized.py:47: test_skip_second_third[6*9-42] 

Before clearing flag:  []

After clearing flag:  []

FAILED

=========== 2 failed, 1 passed, 118 deselected in 0.37 seconds===============