[pytest-dev] plugin status page

[Bruno Oliveira]

Hi Holger, all,

On Sun, Oct 20, 2013 at 11:06 AM, Bruno Oliveira <nicoddemus at gmail.com>wrote:

>
> On Sun, Oct 20, 2013 at 3:52 AM, holger krekel <holger at merlinux.eu> wrote:
>
>>
>> What about generating a "plugin details " page for each plugin so we don't
>> have to think too hard about what to put in the index page?
>>
>
> That's a possibility, for sure.
>
>
>> What is the advantage over just invoking "devpi test PLUGIN-NAME" from the
>> script to automate the download/unpack/test/post-results steps?
>>
>
> I didn't use devpi at that point because of my limited knowledge of devpi
> at the moment, but that is certainly the next step.
>
> I will work on that next, and also work on generating a default tox.ini
> file for plugins that don't have one as you suggested.
>

I have updated the script to create a tox.ini file that just uses "py.test
--help" as a test command for those packages that don't have one. Now the
script tests all pytest plugins in pypi, please take a look at the summary
at the bottom of the page:

https://www.travis-ci.org/nicoddemus/pytest-plugs/jobs/12959702

I guess it would be useful to identify which plugins are using just
"py.test --help" as smoke test and which ones have real tests, since the
former doesn't guarantee that the plugin actually works on that
python/pytest combination.

I looked into devpi to simplify the download/unpack/test process and
stumbled in a minor roadblock: is there anyway to provide a tox.ini file
for those plugins that don't have one? This is the only issue I see right
now with using devpi for this particular application.


>  | > I guess we can implement in devpi a concept similar to travis build
> status
>
>>  > imagens, which we could then use in the plugins_index page.
>> >
>> > Against python 2.6 the script is failing, I have yet to investigate.
>>
>
So, what do you think of the idea of using devpi.net to host test results
and add an endpoint to the API to provide test status images? Or should I
create another app to do this?

Cheers,
Bruno.


> I found the reason for that (http vs https) and now the job runs against
> Python 2.6, 2.7, 3.2 and 3.2.
>
> Thanks!  Got back but caught a flue ...
>>
>
> It happens, hope you get better soon. :)
>
> Cheers,
> Bruno.
>
>
> On Sun, Oct 20, 2013 at 3:52 AM, holger krekel <holger at merlinux.eu> wrote:
>
>> On Mon, Oct 14, 2013 at 23:52 -0300, Bruno Oliveira wrote:
>> > > > >
>> > > > > - i'd collapse NAME and VERSION columns to save space, i.e.
>> > > > >   "pytest-bdd-0.6.1".
>> > > > >
>> > > > > - what about adding download numbers?
>> > > > >
>> > > > > - as to code organisation: you can leave it as is for now or
>> (maybe
>> > > > >   better) put all code and generated files into a dedicated
>> directory
>> > > >
>> > >
>> >
>> > Done, opened a PR for discussion and reviewing. :)
>>
>> only got to it now, looks good so far!
>>
>> > > > > - we should try to collect repository locations. maybe parsing
>> > > > >   for github/bitbucket urls would yield most of them
>> automatically?
>> > > > >   Maybe we need to add some manually.
>> > > > >
>> > > >
>> > > > May I ask why we would need the repository locations? I mean, to
>> work on
>> > > > the compatibility feature we can work directly with packages in
>> pypi or
>> > > > devpi...
>> > >
>> > > purely for the person reading the page and wanting to look at the code
>> > > or do a PR.
>> > >
>> >
>> > Oh I see! Well, parsing the urls seems problematic, because we also
>> have to
>> > take in account the username ("hpk42/pytest" for example). I looked at
>> the
>> > GitHub and BitBucket's rest APIs to see if we could use them for
>> searching,
>> > but it seems only GitHub at the moment has a search API (and in beta
>> stage
>> > at that).
>> >
>> > I think we will have to maintain the repository list manually. Package
>> > authors can help as well by simply opening PRs adding repositories for
>> > their plugins I guess.
>>
>> What about generating a "plugin details " page for each plugin so we don't
>> have to think too hard about what to put in the index page?
>>
>> > > A very interesting bit will be the "2.4.2 compat" and "dev" compat
>> > > > <snip>
>> > >
>> >
>> > Some late night quick hacking and I have come up with this proof of
>> concept:
>> >
>> > https://www.travis-ci.org/nicoddemus/pytest-plugs
>> >
>> > As it is right now it serves only as a prove of concept, of course. The
>> > idea is using travis to handle running things for us, driving it using a
>> > script that downloads and run tests using tox. We can then collect json
>> > information from that and POST those results back to devpi. Also we can
>> use
>> > its build matrix to test against different pytest versions, for example.
>>
>> Hum, interesting.  Is this mainly to make use of travis-ci's build hosts?
>> What is the advantage over just invoking "devpi test PLUGIN-NAME" from the
>> script to automate the download/unpack/test/post-results steps?
>>
>> > I guess we can implement in devpi a concept similar to travis build
>> status
>> > imagens, which we could then use in the plugins_index page.
>> >
>> > Against python 2.6 the script is failing, I have yet to investigate.
>> >
>> > great, i am travelling to Pycon DE tomorrow morning but should be online
>> > >  from time to time.
>> > >
>> >
>> > Nice tripe and good PyCon!
>>
>> Thanks!  Got back but caught a flue ...
>>
>> cheers,
>> holger
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20131023/7e4466f3/attachment-0001.html>