[pytest-dev] plugin status page

[Bruno Oliveira]

Hi Holger,

On Thu, Oct 24, 2013 at 4:52 AM, holger krekel <holger at merlinux.eu> wrote:

> Hi Bruno,
>
> On Wed, Oct 23, 2013 at 21:51 -0200, Bruno Oliveira wrote:
> > Hi Holger, all,
> >
> > <snip>
> > I have updated the script to create a tox.ini file that just uses
> "py.test
> > --help" as a test command for those packages that don't have one. Now the
> > script tests all pytest plugins in pypi, please take a look at the
> summary
> > at the bottom of the page:
> >
> > https://www.travis-ci.org/nicoddemus/pytest-plugs/jobs/12959702
>
> looks good!  I guess having a py27 and py33 row would be great
> to also determine py3 compatibility/installability.
>

Those are already in place as different build jobs, please take a look:

https://www.travis-ci.org/nicoddemus/pytest-plugs

Currently we have py27 and py33 against pytest 2.3.5 and 2.4.2. Adding more
python/pytest versions is easy, we just have to add them to the .travis
file. :)

https://github.com/nicoddemus/pytest-plugs/blob/master/.travis.yml


>
> > I guess it would be useful to identify which plugins are using just
> > "py.test --help" as smoke test and which ones have real tests, since the
> > former doesn't guarantee that the plugin actually works on that
> > python/pytest combination.
> >
> > I looked into devpi to simplify the download/unpack/test process and
> > stumbled in a minor roadblock: is there anyway to provide a tox.ini file
> > for those plugins that don't have one? This is the only issue I see right
> > now with using devpi for this particular application.
>
> Now, but i am going to add a "devpi test -c path-to-ini" option today
> and plan to release devpi-1.2 real soon now.  I'll get back here when it's
> implemented.
>

Perfect, that would greatly simplify things. Let me know if there is
anything I can do to help.

> So, what do you think of the idea of using devpi.net to host test results
> > and add an endpoint to the API to provide test status images? Or should I
> > create another app to do this?
>
> What do you mean with "test status images"?

In general i am open to extending devpi-server to provide more test-status
> info -- just didn't have a chance to think about it in detail yet.
> I guess we could take the plugin testing bits as a starting point.
>

I mean like those badge images served by travis that give the current build
status, for example:

https://secure.travis-ci.org/hpk42/pytest.png

I think we can follow the same approach, generating a badge that gives the
full compatibility status, like a single "py 2.6 2.7 | pytest 2.3.5 |
compatible" image, or even multiple images.

But I guess this is too specific to pytest...

Cheers,
Bruno.


>
> best,
> holger
>
> > Cheers,
> > Bruno.
> >
> >
> > > I found the reason for that (http vs https) and now the job runs
> against
> > > Python 2.6, 2.7, 3.2 and 3.2.
> > >
> > > Thanks!  Got back but caught a flue ...
> > >>
> > >
> > > It happens, hope you get better soon. :)
> > >
> > > Cheers,
> > > Bruno.
> > >
> > >
> > > On Sun, Oct 20, 2013 at 3:52 AM, holger krekel <holger at merlinux.eu>
> wrote:
> > >
> > >> On Mon, Oct 14, 2013 at 23:52 -0300, Bruno Oliveira wrote:
> > >> > > > >
> > >> > > > > - i'd collapse NAME and VERSION columns to save space, i.e.
> > >> > > > >   "pytest-bdd-0.6.1".
> > >> > > > >
> > >> > > > > - what about adding download numbers?
> > >> > > > >
> > >> > > > > - as to code organisation: you can leave it as is for now or
> > >> (maybe
> > >> > > > >   better) put all code and generated files into a dedicated
> > >> directory
> > >> > > >
> > >> > >
> > >> >
> > >> > Done, opened a PR for discussion and reviewing. :)
> > >>
> > >> only got to it now, looks good so far!
> > >>
> > >> > > > > - we should try to collect repository locations. maybe parsing
> > >> > > > >   for github/bitbucket urls would yield most of them
> > >> automatically?
> > >> > > > >   Maybe we need to add some manually.
> > >> > > > >
> > >> > > >
> > >> > > > May I ask why we would need the repository locations? I mean, to
> > >> work on
> > >> > > > the compatibility feature we can work directly with packages in
> > >> pypi or
> > >> > > > devpi...
> > >> > >
> > >> > > purely for the person reading the page and wanting to look at the
> code
> > >> > > or do a PR.
> > >> > >
> > >> >
> > >> > Oh I see! Well, parsing the urls seems problematic, because we also
> > >> have to
> > >> > take in account the username ("hpk42/pytest" for example). I looked
> at
> > >> the
> > >> > GitHub and BitBucket's rest APIs to see if we could use them for
> > >> searching,
> > >> > but it seems only GitHub at the moment has a search API (and in beta
> > >> stage
> > >> > at that).
> > >> >
> > >> > I think we will have to maintain the repository list manually.
> Package
> > >> > authors can help as well by simply opening PRs adding repositories
> for
> > >> > their plugins I guess.
> > >>
> > >> What about generating a "plugin details " page for each plugin so we
> don't
> > >> have to think too hard about what to put in the index page?
> > >>
> > >> > > A very interesting bit will be the "2.4.2 compat" and "dev" compat
> > >> > > > <snip>
> > >> > >
> > >> >
> > >> > Some late night quick hacking and I have come up with this proof of
> > >> concept:
> > >> >
> > >> > https://www.travis-ci.org/nicoddemus/pytest-plugs
> > >> >
> > >> > As it is right now it serves only as a prove of concept, of course.
> The
> > >> > idea is using travis to handle running things for us, driving it
> using a
> > >> > script that downloads and run tests using tox. We can then collect
> json
> > >> > information from that and POST those results back to devpi. Also we
> can
> > >> use
> > >> > its build matrix to test against different pytest versions, for
> example.
> > >>
> > >> Hum, interesting.  Is this mainly to make use of travis-ci's build
> hosts?
> > >> What is the advantage over just invoking "devpi test PLUGIN-NAME"
> from the
> > >> script to automate the download/unpack/test/post-results steps?
> > >>
> > >> > I guess we can implement in devpi a concept similar to travis build
> > >> status
> > >> > imagens, which we could then use in the plugins_index page.
> > >> >
> > >> > Against python 2.6 the script is failing, I have yet to investigate.
> > >> >
> > >> > great, i am travelling to Pycon DE tomorrow morning but should be
> online
> > >> > >  from time to time.
> > >> > >
> > >> >
> > >> > Nice tripe and good PyCon!
> > >>
> > >> Thanks!  Got back but caught a flue ...
> > >>
> > >> cheers,
> > >> holger
> > >>
> > >
> > >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20131024/37d8366e/attachment-0001.html>