[pytest-dev] New plugin for generating HTML report (and request for help)

holger krekel holger at merlinux.eu
Sun Apr 19 10:16:36 CEST 2015 

On Thu, Apr 16, 2015 at 17:46 +0100, Dave Hunt wrote:
> Thanks Bruno, the custom hook works well. Here’s the change: https://github.com/davehunt/pytest-html/commit/2a405a3cdc638c1896ab3d1074296496bb1fa3a8 <https://github.com/davehunt/pytest-html/commit/2a405a3cdc638c1896ab3d1074296496bb1fa3a8>
> 
> Now I need to work out why I get issues when I use this with pytest-xdist. It looks like it’s complaining about the extra report details not being serializable?! See traceback below:

Yes, you need to make sure that anything extra you put on the report is marshallable.
In the case of your url you may just send a string and parse as url on
the logreport/master side.

holger


> test_login.py::TestLogin::test_invalid_username INTERNALERROR> Traceback (most recent call last):
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/main.py", line 84, in wrap_session
> INTERNALERROR>     doit(config, session)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/main.py", line 122, in _main
> INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 521, in __call__
> INTERNALERROR>     return self._docall(self.methods, kwargs)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 528, in _docall
> INTERNALERROR>     firstresult=self.firstresult).execute()
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 394, in execute
> INTERNALERROR>     res = method(*args)
> INTERNALERROR>   File "<remote exec>", line 56, in pytest_runtestloop
> INTERNALERROR>   File "<remote exec>", line 72, in run_tests
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 521, in __call__
> INTERNALERROR>     return self._docall(self.methods, kwargs)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 528, in _docall
> INTERNALERROR>     firstresult=self.firstresult).execute()
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 393, in execute
> INTERNALERROR>     return wrapped_call(method(*args), self.execute)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 113, in wrapped_call
> INTERNALERROR>     return call_outcome.get_result()
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 138, in get_result
> INTERNALERROR>     py.builtin._reraise(*ex)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 123, in __init__
> INTERNALERROR>     self.result = func()
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 394, in execute
> INTERNALERROR>     res = method(*args)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/runner.py", line 65, in pytest_runtest_protocol
> INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/runner.py", line 75, in runtestprotocol
> INTERNALERROR>     reports.append(call_and_report(item, "call", log))
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/runner.py", line 123, in call_and_report
> INTERNALERROR>     hook.pytest_runtest_logreport(report=report)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 521, in __call__
> INTERNALERROR>     return self._docall(self.methods, kwargs)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 528, in _docall
> INTERNALERROR>     firstresult=self.firstresult).execute()
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 394, in execute
> INTERNALERROR>     res = method(*args)
> INTERNALERROR>   File "<remote exec>", line 86, in pytest_runtest_logreport
> INTERNALERROR>   File "<remote exec>", line 23, in sendevent
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 691, in send
> INTERNALERROR>     self.gateway._send(Message.CHANNEL_DATA, self.id, dumps_internal(item))
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1295, in dumps_internal
> INTERNALERROR>     return _Serializer().save(obj)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1313, in save
> INTERNALERROR>     self._save(obj)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1331, in _save
> INTERNALERROR>     dispatch(self, obj)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1412, in save_tuple
> INTERNALERROR>     self._save(item)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1331, in _save
> INTERNALERROR>     dispatch(self, obj)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1408, in save_dict
> INTERNALERROR>     self._write_setitem(key, value)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1402, in _write_setitem
> INTERNALERROR>     self._save(value)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1331, in _save
> INTERNALERROR>     dispatch(self, obj)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1408, in save_dict
> INTERNALERROR>     self._write_setitem(key, value)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1402, in _write_setitem
> INTERNALERROR>     self._save(value)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1331, in _save
> INTERNALERROR>     dispatch(self, obj)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1398, in save_list
> INTERNALERROR>     self._write_setitem(i, item)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1402, in _write_setitem
> INTERNALERROR>     self._save(value)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/execnet/gateway_base.py", line 1329, in _save
> INTERNALERROR>     raise DumpError("can't serialize %s" % (tp,))
> INTERNALERROR> DumpError: can't serialize <class 'pytest_html.URL'>
> 
> INTERNALERROR> Traceback (most recent call last):
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/main.py", line 84, in wrap_session
> INTERNALERROR>     doit(config, session)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/main.py", line 122, in _main
> INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 521, in __call__
> INTERNALERROR>     return self._docall(self.methods, kwargs)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 528, in _docall
> INTERNALERROR>     firstresult=self.firstresult).execute()
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/pytest-2.7.0-py2.7.egg/_pytest/core.py", line 394, in execute
> INTERNALERROR>     res = method(*args)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/xdist/dsession.py", line 504, in pytest_runtestloop
> INTERNALERROR>     self.loop_once()
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/xdist/dsession.py", line 522, in loop_once
> INTERNALERROR>     call(**kwargs)
> INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-97b1b20a17ffb8d0/lib/python2.7/site-packages/xdist/dsession.py", line 560, in slave_slavefinished
> INTERNALERROR>     assert not crashitem, (crashitem, node)
> INTERNALERROR> AssertionError: ('test_login.py::TestLogin::()::test_invalid_username', <SlaveController gw0>)
> 
> --
> Dave Hunt
> Firefox OS Automation Engineer
> Mozilla Corporation
> dhunt at mozilla.com
> 
> > On 15 Apr 2015, at 21:16, Bruno Oliveira <nicoddemus at gmail.com> wrote:
> > 
> > Hi Dave,
> > 
> > Congratulations on this initiative to separate this functionality into a separate plugin, it seems very useful! :)
> > 
> > One way to allow other plugins to interact with yours is by defining your own hooks. Pytest-xdist does this[1], and as an example, the builtin hook pytest_report_header[2] seems to do something similar to what you want, in the sense that it allows other plugins to add information to the terminal header.
> > 
> > Perhaps you can declare a hook `pytest_html_report_environment`, which plugins can implement and return a dict of environment variables to be added to the report? Or perhaps, if it makes sense for the environment details to appear in the terminal output as well, you could in fact just call `pytest_report_header` and add its contents into the HTML; the latter has the benefit that existing plugins which already use `pytest_report_header` will be able to write their information to the HTML report.
> > 
> > Cheers,
> > 
> > [1] https://bitbucket.org/pytest-dev/pytest-xdist/src/00cfff4834e718fd3c1ccec40811e734d796f631/xdist/newhooks.py <https://bitbucket.org/pytest-dev/pytest-xdist/src/00cfff4834e718fd3c1ccec40811e734d796f631/xdist/newhooks.py>
> > [2] http://pytest.org/latest/example/simple.html?highlight=pytest_report_header#adding-info-to-test-report-header <http://pytest.org/latest/example/simple.html?highlight=pytest_report_header#adding-info-to-test-report-header>
> > 
> > On Wed, Apr 15, 2015 at 12:05 PM, Dave Hunt <dhunt at mozilla.com <mailto:dhunt at mozilla.com>> wrote:
> > Hey py.testers!
> > 
> > I’ve been working on pulling a feature out of our pytest-mozwebqa plugin into a standalone plugin. I think others may find it useful, and it would simplify maintenance of the former plugin to split this out. Please take a look over the pytest-html plugin, the source code is available here: https://github.com/davehunt/pytest-html <https://github.com/davehunt/pytest-html>
> > 
> > Basically this plugin provides a new command line option of —html which allows the user to specify a path for a HTML report to be generated. The report can then be enhanced by adding extra components via  the pytest_runtest_makereport hook. You can see an example of this in my in-progress branch of pytest-mozwebqa where I’ve switched to using the new plugin: https://github.com/davehunt/pytest-mozwebqa/blob/1074f0770a146cff3108191a2fe239d15cfd92e4/pytest_mozwebqa/pytest_mozwebqa.py#L126 <https://github.com/davehunt/pytest-mozwebqa/blob/1074f0770a146cff3108191a2fe239d15cfd92e4/pytest_mozwebqa/pytest_mozwebqa.py#L126>
> > 
> > What I’d like some help with is providing the HTML report with environment details. This is essentially just a dictionary that will be included in the HTML report, but should be optionally provided by another plugin or conftest.py file. At the moment I accept an environment keyword argument when instantiating the HTMLReport object, but I’m not aware of a way to influence this from another plugin. See: https://github.com/davehunt/pytest-html/blob/752f229b990c80e66195374a4ed4fe22b98017c6/pytest_html.py#L44 <https://github.com/davehunt/pytest-html/blob/752f229b990c80e66195374a4ed4fe22b98017c6/pytest_html.py#L44> and https://github.com/davehunt/pytest-html/blob/752f229b990c80e66195374a4ed4fe22b98017c6/pytest_html.py#L195 <https://github.com/davehunt/pytest-html/blob/752f229b990c80e66195374a4ed4fe22b98017c6/pytest_html.py#L195>
> > 
> > If anybody has any advice it would be much appreciated!
> > 
> > Thanks in advance,
> > 
> > --
> > Dave Hunt
> > Automation Engineer
> > Mozilla Corporation
> > dhunt at mozilla.com <mailto:dhunt at mozilla.com>
> > 
> > _______________________________________________
> > pytest-dev mailing list
> > pytest-dev at python.org <mailto:pytest-dev at python.org>
> > https://mail.python.org/mailman/listinfo/pytest-dev <https://mail.python.org/mailman/listinfo/pytest-dev>
> > 
> > 
> 

> _______________________________________________
> pytest-dev mailing list
> pytest-dev at python.org
> https://mail.python.org/mailman/listinfo/pytest-dev


-- 
about me:    http://holgerkrekel.net/about-me/
contracting: http://merlinux.eu

More information about the pytest-dev mailing list