[pytest-dev] Custom reporting for asserts without comparison operators?
Ronny Pfannschmidt
rpfannsc at redhat.com
Thu Mar 22 04:12:56 EDT 2018
that approach is broken in the sense, that it breaks behaviour expectations,
an return value helper, that triggers an assertion on its own is simply no
longer a return value helper, but a assertion helper
supporting it like that would result in a really bad api
instead having assertion helper that returns a "truthy" object which can
be introspected by pytest and/or negated should be more suitable
2018-03-22 3:39 GMT+01:00 Shawn Brown <03sjbrown at gmail.com>:
> Ah. It's good to see that this has been thought about before.
>
> My motivation for asking this question was to perform my due diligence and
> make sure I wasn't missing something before moving ahead. My immediate
> need is handled by using assert_myfunc() to raise its own error
> internally--same as Floris' example. Though, it's not ideal.
>
> I know my examples have been vague as I've stripped the specifics of my
> project to focus the question specifically on pytest's behavior and I
> greatly appreciate everyone who is giving some thought to this.
>
> As Ronny mentioned, I'm sure it's possible to address this without
> user-facing AST manipulation. But I'm not familiar enough with the code
> base to see where I can best hack on the representations. However, I do
> have a working AST-based demonstration (below). This uses a fragile
> monkey-patch that is just asking for trouble so please take this for the
> experimental hack it is...
>
>
> FILE "conftest.py":
>
> import ast
> import _pytest
>
> def my_ast_prerewrite_hook(ast_assert):
> """Modifies AST of certain asserts before pytest-rewriting."""
> # Demo AST-tree manipulation (actual implemenation
> # would need to be more careful than this).
> if (isinstance(ast_assert.test, ast.Call)
> and isinstance(ast_assert.test.func, ast.Name)
> and ast_assert.test.func.id == 'myfunc'):
>
> ast_assert.test.func = ast.Name('assert_myfunc', ast.Load())
>
> return ast_assert
>
> # UNDESIRABLE MONKEY PATCHING!!!
> class ModifiedRewriter(_pytest.assertion.rewrite.AssertionRewriter):
> def visit_Assert(self, assert_):
> assert_ = my_ast_prerewrite_hook(assert_) # <- PRE-REWRITE
> HOOK
> return super(ModifiedRewriter, self).visit_Assert(assert_)
>
> def rewrite_asserts(mod, module_path=None, config=None):
> ModifiedRewriter(module_path, config).run(mod)
>
> _pytest.assertion.rewrite.rewrite_asserts = rewrite_asserts
>
>
> FILE "test_ast_hook_approach.py":
>
> import pytest
>
> # Test helpers.
> def myfunc(x):
> return x == 42
>
> def assert_myfunc(x):
> __tracebackhide__ = True
> if not myfunc(x):
> msg = 'custom report\nmulti-line output\nmyfunc({0}) failed'
> raise AssertionError(msg.format(x))
> return True
>
> # Test cases.
> def test_1passing():
> assert myfunc(42)
>
> def test_2passing():
> assert myfunc(41) is False
>
> def test_3passing():
> with pytest.raises(AssertionError) as excinfo:
> assert myfunc(41)
> assert 'custom report' in str(excinfo.value)
>
> def test_4failing():
> assert myfunc(41)
>
>
> Running the above test gives 3 passing cases and 1 failing case (which
> uses the custom report). Also, test_2passing() checks for "is False"
> instead of just "== False" which I think would be wonderful to support as
> it removes all caveats for the user (so users get a real False when they
> expect False, instead of a Falsey alternative). Also, if I were going to
> use AST manipulation like this, I would probably reference assert_myfunc()
> by attaching it as a private attribute to myfunc() itself -- and then
> reference it with ast.Attribute() node instead of an ast.Name(). But again,
> solving this without AST manipulation could be better in many ways.
>
> --Shawn
>
>
> On Mon, Mar 19, 2018 at 1:59 PM, Ronny Pfannschmidt <
> ich at ronnypfannschmidt.de> wrote:
>
>> hi everyone,
>>
>> this is just about single value assertion helpers
>>
>> i logged an feature request about that a few year back
>> see https://github.com/pytest-dev/pytest/issues/95 -
>>
>> so basically this use-case was known since 2011 ^^ and doesn't require
>> ast rewriting lice macros,
>> just proper engineering of the representation and handling of single
>> values in the assertion rewriter.
>>
>> -- Ronny
>>
>>
>> Am 19.03.2018 um 15:13 schrieb holger krekel:
>> > On Mon, Mar 19, 2018 at 15:03 +0100, Floris Bruynooghe wrote:
>> >> On Sun, Mar 18 2018, Shawn Brown wrote:
>> >>> Unfortunately, this does not solve my usecase. I'm trying to handle
>> cases
>> >>> where the following statement would pass:
>> >>>
>> >>> assert myfunc(failing_input) == False
>> >>>
>> >>> But where this next statement would fail using my custom report:
>> >>>
>> >>> assert myfunc(failing_input)
>> >>>
>> >>> Calling myfunc() needs to return True or False (or at least Truthy or
>> >>> Falsy)--this is locked-in behavior.
>> >> I'm not sure if this is compatible with Python's semantics really. If
>> I
>> >> understand correctly you're asking for a full-on macro implementation
>> on
>> >> Python or something. Which in theory you could do with an AST
>> >> NodeVisitor, but really Python isn't made for this -- sounds like you'd
>> >> enjoy lisp! ;-)
>> >>
>> >> The best thing I can suggest is to make use of the::
>> >>
>> >> assert myfunc(failing_input), repr(myfunc(failing_input()))
>> > i wonder if one could try to rewrite the ast for "assert myfunc(x)" to
>> > "assert __pytest_funcrepr_helper(myfunc(x), 'myfunc(x)')" with
>> something like:
>> >
>> > class __pytest_funcrepr_helper:
>> > def __init__(self, val, source):
>> > self.val = val
>> > self.source = source
>> > def __bool__(self):
>> > return bool(self.val)
>> > def __repr__(self):
>> > return "{!r} returned non-true {!r}".format(self.source,
>> self.val)
>> >
>> > but maybe i am not grasping all details involved. It's been a while
>> since
>> > i looked into ast-rewriting ...
>> >
>> > holger
>> >
>> >
>> >> functionality to also get a custom error message. Here your myfunc()
>> >> whould have to return some object which both implements __bool__ as
>> well
>> >> as __repr__ I guess.
>> >>
>> >> Maybe there's a feature request in here for something like this::
>> >>
>> >> class Foo:
>> >> def __bool__(self):
>> >> return False
>> >>
>> >> def __repr__(self):
>> >> return 'multiline\nstring'
>> >>
>> >> assert Foo()
>> >>
>> >> To actually show the repr in the error message, which it currently
>> >> doesn't. I'd like to know what other people think of such a feature
>> >> though, and haven't thought through all the implications yet. But I'm
>> >> curious, would something like that solve your case?
>> >>
>> >> Cheers,
>> >> Floris
>> >> _______________________________________________
>> >> pytest-dev mailing list
>> >> pytest-dev at python.org
>> >> https://mail.python.org/mailman/listinfo/pytest-dev
>> > _______________________________________________
>> > pytest-dev mailing list
>> > pytest-dev at python.org
>> > https://mail.python.org/mailman/listinfo/pytest-dev
>>
>
> _______________________________________________
> pytest-dev mailing list
> pytest-dev at python.org
> https://mail.python.org/mailman/listinfo/pytest-dev
>
>
--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael
O'Neill, Eric Shander
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20180322/74447e54/attachment.html>
More information about the pytest-dev
mailing list