[New-bugs-announce] [issue32213] assertRaises and subTest context managers cannot be nested

[Paul Ganssle]

New submission from Paul Ganssle <p.ganssle at gmail.com>:

The TestCase.assertRaises and TestCase.subTest macros apparently don't interact with each other well. To demonstrate, consider the following code:

    from unittest import TestCase

    class SubTestRaisesTest(TestCase):
        def test_assert_outer(self):
            for to_raise in [True, False]:
                with self.assertRaises(Exception):
                    with self.subTest(to_raise=to_raise):
                        if to_raise:
                            raise Exception()

        def test_assert_inner(self):
            for to_raise in [True, False]:
                with self.subTest(to_raise=to_raise):
                    with self.assertRaises(Exception):
                        if to_raise:
                            raise Exception()

This actually fails in two different ways.

For test_assert_outer:
  
-with subtest `to_raise=True`, the test (correctly) passes.
-with subtest `to_raise=False`, the test (correctly) fails, but the subtest is not actually assigned (no indication of which subtest it was that failed):



    ======================================================================
    FAIL: test_assert_outer (test_bug.SubTestRaisesTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File ".../assert_demo/test_bug.py", line 9, in test_assert_outer
        raise Exception()
    AssertionError: Exception not raised


For test_assert_inner:
- with subtest `to_raise=False`, the test (corrrectly) fails, *and* the subtest is set correctly:


    ======================================================================
    FAIL: test_assert_inner (test_bug.SubTestRaisesTest) (to_raise=False)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "..../assert_demo/test_bug.py", line 16, in test_assert_inner
        raise Exception()
    AssertionError: Exception not raised


- with subtest `to_raise=False` the test (incorrectly) fails as an error, because the exception is never caught:


    ======================================================================
    ERROR: test_assert_outer (test_bug.SubTestRaisesTest) (to_raise=True)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "..../assert_demo/test_bug.py", line 9, in test_assert_outer
        raise Exception()
    Exception


So, to sum up, the behavior that needs to be fixed:

1. When assertRaises is the outer context, the subtest value needs to be set for the failing tests.

2. When assertRaises is the *inner* context, the exception needs to be caught properly and cleared on exit from the assertRaises context.

----------
components: Tests
messages: 307569
nosy: p-ganssle
priority: normal
severity: normal
status: open
title: assertRaises and subTest context managers cannot be nested
versions: Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32213>
_______________________________________