[pytest-dev] (new) pytest plugins / note to plugin authors

[holger krekel]

Hi Bruno,

On Fri, Oct 04, 2013 at 18:06 -0300, Bruno Oliveira wrote:
> Hi Holger,
> 
> On Fri, Oct 4, 2013 at 4:45 PM, holger krekel <holger at merlinux.eu> wrote:
> 
> > Hi Bruno,
> >
> > On Fri, Oct 04, 2013 at 12:43 -0300, Bruno Oliveira wrote:
> > > Hi Holger,
> > > Did you give any thoughts on how this could be done?
> >
> > note sure in detail but i guess each plugin could be presented
> > with this information:
> >
> > pluginname and latest version, pypi link, maintainer/author, repository
> > link,
> > one-line description, test status py27/py33
> >
> 
> Are you interested in displaying compatibility to several py.test versions,
> or only to the latest one?

ideally latest release and trunk ...

> and some way to re-generate this information (especially the test status).
> > I guess we could start with considering plugins that have a tox.ini
> > and then write a script to download the plugin, unpack it, run tests etc.
> >
> 
> I considered this, but then I realize we might have to implement several
> features to support that, like installing dependencies: pytest-qt for
> instance requires PySide, which must be installed for it to run its own
> tests. Other plugins might have other requirements to run its tests (like
> having a browser open for headless testing), which would have to be
> supported as well and so on. I fear we would have to implement a lot of
> infrastructure similar to what travis-ci.org or d
> <http://crate.io>rone.iodoes for example.
> 
> Perhaps we could adopt a solution similar to coveralls.io, in which the
> author is responsible to upload coverage data, and the site collects and
> displays that information. We could provide a script or code snippet to be
> included by the plugin author in he's C.I of choice that would generate and
> upload pytest-compatibility information. This could be easily hosted in a
> PaaS like Heroku.
> 
> We would have to count on plugin authors' contribution to the system for it
> to work on this scheme, of course.

I wonder how difficult most plugins really are to setup.
PyQT might be difficult but OTOH many others should just be fine.

And pushing the responsibility for performing tests and uploading results
still would not get the benefit of testing current plugin releases
against a new core pytest release (which can break many plugins whereas
a new plugin release can just break that plugin :).

So i'd think it's best to check how many of the plugins even provide tests.

But even just collecting the other information i mentioned (without
the test status / running questions) would already be helpful, btw.

cheers,
holger

> The other part is how to present this info, but e.g. a CSV file
> > can be rendered by sphinx and the main docs could include it
> > or we make a separate page if need be.
> 
> 
> > thanks for considering to help!
> >
> 
> Glad to. :)
> 
> Best Regards,
> Bruno
> 
> >
> > holger
> >
> > > Cheers,
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Fri, Oct 4, 2013 at 10:09 AM, holger krekel <holger at merlinux.eu>
> > wrote:
> > >
> > > >
> > > > I just did three pytest plugin releases to get out a few minor
> > > > fixes and improvements. see below.  As far as i am aware most
> > > > existing plugins (as well as the new releases here) should be
> > > > compatible to pytest-2.4.2.  Some plugins like pytest-capturelog
> > > > have seen their latest release three years ago but remain functional.
> > > > A good sign that the plugin system is reasonable i guess.
> > > >
> > > > If someone wants to help with organizing plugin compatibility
> > > > tests and maintaining a status page, i'd really appreciate that.
> > > > With over 50 existing plugins it's not easy to stay on top.
> > > > The current pytest.org page is only a start:
> > > >
> > > >     http://pytest.org/latest/plugins.html#extplugins
> > > >
> > > > and here is the full pypi list:
> > > >
> > > >
> > > >
> > https://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search
> > > >
> > > > there are also some github/bitbucket-only plugins, in addition.
> > > >
> > > > And finally a note to plugin authors: please be subscribed
> > > > to pytest-dev and post new plugin releases here.
> > > >
> > > > best,
> > > > holger
> > > >
> > > > pytest-xdist-1.9:
> > > >
> > > > - changed LICENSE to MIT
> > > >
> > > > - fix duplicate reported test ids with --looponfailing
> > > >   (thanks Jeremy Thurgood)
> > > >
> > > > - fix pytest issue41: re-run tests on all file changes, not just
> > > >   randomly select ones like .py/.c.
> > > >
> > > > - fix pytest issue347: slaves running on top of Python3.2
> > > >   will set PYTHONDONTWRITEYBTECODE to 1 to avoid import concurrency
> > > >   bugs.
> > > >
> > > > pytest-xprocess-0.8:
> > > >
> > > > - support python3 (tested on linux/win32)
> > > >
> > > > - split out pytest independent process support into xprocess.py
> > > >
> > > > - add info.kill() method and some smaller refactoring
> > > >
> > > > - fix various windows related Popen / killing details
> > > >
> > > > - add tests
> > > >
> > > >
> > > > pytest-pep8:
> > > >
> > > > - use pytest-2.4.2 node.add_marker() API for adding "pep8" marker
> > > >
> > > > _______________________________________________
> > > > Pytest-dev mailing list
> > > > Pytest-dev at python.org
> > > > https://mail.python.org/mailman/listinfo/pytest-dev
> > > >
> >