[pytest-dev] Custom reporting for asserts without comparison operators?
RonnyPfannschmidt
opensource at ronnypfannschmidt.de
Thu Mar 22 18:45:35 EDT 2018
i don't see under what criteria that test is sensible in terms of
integration,
if the helper does the assert, you do not need a assert statement
if the helper does not do the assert, the python assertion mechanism on
its own is unable to provide the meta-data
as such that test is really only able to test a non-integrated solution
and i don't consider it to be of value when talking about integration
-- Ronny
Am 22.03.2018 um 16:35 schrieb Shawn Brown:
> Would a return value helper--as you are thinking about it--be able to
> handle cases like test_3passing()?
>
> def test_3passing():
> with pytest.raises(AssertionError) as excinfo:
> assert myfunc(41)
> assert 'custom report' in str(excinfo.value)
>
> I've looked around the code base but I'm not sure where best to hack
> on things. I've played with changes to
> _pytest.assertion.rewrite._saferepr() but this doesn't seem to be the
> right place to address this sort of change (newline handling and other
> details are handled elsewhere).
>
>
> On Thu, Mar 22, 2018 at 4:12 AM, Ronny Pfannschmidt
> <rpfannsc at redhat.com <mailto:rpfannsc at redhat.com>> wrote:
>
> that approach is broken in the sense, that it breaks behaviour
> expectations,
> an return value helper, that triggers an assertion on its own is
> simply no longer a return value helper, but a assertion helper
>
> supporting it like that would result in a really bad api
>
> instead having assertion helper that returns a "truthy" object
> which can be introspected by pytest and/or negated should be more
> suitable
>
> 2018-03-22 3:39 GMT+01:00 Shawn Brown <03sjbrown at gmail.com
> <mailto:03sjbrown at gmail.com>>:
>
> Ah. It's good to see that this has been thought about before.
>
> My motivation for asking this question was to perform my due
> diligence and make sure I wasn't missing something before
> moving ahead. My immediate need is handled by using
> assert_myfunc() to raise its own error internally--same as
> Floris' example. Though, it's not ideal.
>
> I know my examples have been vague as I've stripped the
> specifics of my project to focus the question specifically on
> pytest's behavior and I greatly appreciate everyone who is
> giving some thought to this.
>
> As Ronny mentioned, I'm sure it's possible to address this
> without user-facing AST manipulation. But I'm not familiar
> enough with the code base to see where I can best hack on the
> representations. However, I do have a working AST-based
> demonstration (below). This uses a fragile monkey-patch that
> is just asking for trouble so please take this for the
> experimental hack it is...
>
>
> FILE "conftest.py":
>
> import ast
> import _pytest
>
> def my_ast_prerewrite_hook(ast_assert):
> """Modifies AST of certain asserts before
> pytest-rewriting."""
> # Demo AST-tree manipulation (actual implemenation
> # would need to be more careful than this).
> if (isinstance(ast_assert.test, ast.Call)
> and isinstance(ast_assert.test.func, ast.Name)
> and ast_assert.test.func.id
> <http://ast_assert.test.func.id> == 'myfunc'):
>
> ast_assert.test.func = ast.Name('assert_myfunc',
> ast.Load())
>
> return ast_assert
>
> # UNDESIRABLE MONKEY PATCHING!!!
> class
> ModifiedRewriter(_pytest.assertion.rewrite.AssertionRewriter):
> def visit_Assert(self, assert_):
> assert_ = my_ast_prerewrite_hook(assert_) # <-
> PRE-REWRITE HOOK
> return super(ModifiedRewriter,
> self).visit_Assert(assert_)
>
> def rewrite_asserts(mod, module_path=None, config=None):
> ModifiedRewriter(module_path, config).run(mod)
>
> _pytest.assertion.rewrite.rewrite_asserts = rewrite_asserts
>
>
> FILE "test_ast_hook_approach.py":
>
> import pytest
>
> # Test helpers.
> def myfunc(x):
> return x == 42
>
> def assert_myfunc(x):
> __tracebackhide__ = True
> if not myfunc(x):
> msg = 'custom report\nmulti-line
> output\nmyfunc({0}) failed'
> raise AssertionError(msg.format(x))
> return True
>
> # Test cases.
> def test_1passing():
> assert myfunc(42)
>
> def test_2passing():
> assert myfunc(41) is False
>
> def test_3passing():
> with pytest.raises(AssertionError) as excinfo:
> assert myfunc(41)
> assert 'custom report' in str(excinfo.value)
>
> def test_4failing():
> assert myfunc(41)
>
>
> Running the above test gives 3 passing cases and 1 failing
> case (which uses the custom report). Also, test_2passing()
> checks for "is False" instead of just "== False" which I think
> would be wonderful to support as it removes all caveats for
> the user (so users get a real False when they expect False,
> instead of a Falsey alternative). Also, if I were going to use
> AST manipulation like this, I would probably reference
> assert_myfunc() by attaching it as a private attribute to
> myfunc() itself -- and then reference it with ast.Attribute()
> node instead of an ast.Name(). But again, solving this without
> AST manipulation could be better in many ways.
>
> --Shawn
>
>
> On Mon, Mar 19, 2018 at 1:59 PM, Ronny Pfannschmidt
> <ich at ronnypfannschmidt.de <mailto:ich at ronnypfannschmidt.de>>
> wrote:
>
> hi everyone,
>
> this is just about single value assertion helpers
>
> i logged an feature request about that a few year back
> see https://github.com/pytest-dev/pytest/issues/95
> <https://github.com/pytest-dev/pytest/issues/95> -
>
> so basically this use-case was known since 2011 ^^ and
> doesn't require
> ast rewriting lice macros,
> just proper engineering of the representation and handling
> of single
> values in the assertion rewriter.
>
> -- Ronny
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20180322/0cb605fa/attachment-0001.html>
More information about the pytest-dev
mailing list