[py-dev] Advanced assert equal

[Floris Bruynooghe]

Hello Holger

On Thu, Sep 02, 2010 at 11:27:22AM +0200, holger krekel wrote:
> would have the AST-transformation call back into a generic
> pytest-hook if an assertion fails.  So one could customize
> "hasattr(x, y)" or "x in y" or "x == y == z" etc. and all
> such representation code would move to the py/_plugin/pytest_assertion.py 
> plugin. That being said going for just comparisons right now
> is fine, let's not have more general thoughts hold us up.

I wouldn't want to make this more difficult or incompatible though, so
I'd be interested in how you where thinking of doing this.  I guess
this would be exposing the various .visit_*() methods in
_assertionnew.py in some way?  If so I think there's also value in the
more simple pytest_assert_compare(op, left, right) I'm proposing here
(and others like "in", "is" etc) as they don't require having any AST
knowledge to implement, at a cost of slightly less flexibility.

> On Mon, Aug 16, 2010 at 14:51 +0100, Floris Bruynooghe wrote:
> > A possible api for the hook
> > could be:
> > 
> > def pytest_assert_compare(op, left, right, left_repr, right_repr):
> >     """Customise compare
> > 
> >     Return None for no custom compare, otherwise return a list of
> >     strings.  The strings will be joined by newlines but any newlines
> >     *in* a string will be escaped.
> >     """
> >     pass
> > 
> > I guess the reprs are not really necessary if there's an easy way to
> > make them.  It's just that I used them in my original patch.
> 
> Hum, couldn't custom representations just mean that there is no 
> "history" of where the object comes from? 
> 
> The hook looks otherwise good to me.

I've made this hook pytest_assert_compare(op, left, right) now, as
I've found py.io.saferepr() which seems to do a good job.

Not sure what you're referring too with the "history" comment.  If you
mean that as soon as a specialised hook is found the previous
explanations (e.g. from Assert AST node) get lost I'd like to
disagree.  It seems reasonable for the new hooks to just provide a
more detailed/specialised explanation of a particular part of the
failure rather then replace the entire explanation.  Otherwise you
might also lose detail.

> (I guess you are aware that any pytest-hook implementation can always
> choose to accept less than the available arguments).

I wasn't actually, neat.

> > There's also the question of who should truncate large amounts of data
> > (e.g. screenfulls of diffs): the hook itself, the caller of the hook
> > or _format_explanation()?  Probably one of the first two to get rid of
> > the memory usage as soon as possible.
> 
> If a hook returns something we should (probably) not further do anything
> with it in _format_explanation().  And making truncation the repsonsibility 
> of the hook makes sense i think.

ok

> I am not sure about the general use cases, from my side:
> 
>     x == y
>     x != y
>     x in y
>    
> are the interesting ones (with some of the objects being long
> lists, long strings etc.). so some hook for a "binary" relation
> makes sense, pytest_assert_binrepr(op) where op could be "==",
> "in" or "is" etc.

To me it seems more logical to add a separate hook for each .visit_*()
method rather then put multiple together.  But maybe that seems
artificial from a user point of view?

> Floris, i'd very much like to start using/having improved assertion
> representations.  Happy to review patches or forks for inclusion.

I've attached a new patch in which I attempt to use the hook system
and added a pytest_assert_comare(op, left, right) hook.  I must admit
I don't fully understand the plugin/hook system so hope I did it
right [0].  Again I've not concentrated on the actual specific
comparisons, rather would like to get a reasonable idea of how good
the general approach is.

If you like this version I can create a fork on bitbucket and start
working on more/better hook implementations.


Regards
Floris


[0] I assume that py.test.config.hook.pytest_assert_compare is the
    function to call and that it returns a list with the results of
    each such function found, with the first element being the most
    "specific" result.  But I just figured that out using trial and
    error rather then understand the plugin system.

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pytest_assert.diff
Type: text/x-diff
Size: 8410 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20100905/f743f0e8/attachment.diff>