From prologic at shortcircuit.net.au  Tue Mar  1 02:57:05 2011
From: prologic at shortcircuit.net.au (James Mills)
Date: Tue, 1 Mar 2011 11:57:05 +1000
Subject: [py-dev] pytest for python 3 ?
Message-ID: <AANLkTinqsZL-=+MPcR5C4H-r5dBBfvagP0f0bSimGxjq@mail.gmail.com>

Has this happened yet ?

We want to use Python 3.2 at work (on our product)
and also pytest (it's the best testing framework!)

cheers
James

-- 
-- James Mills
--
-- "Problems are solved by method"


From flub at devork.be  Tue Mar  1 09:50:11 2011
From: flub at devork.be (Floris Bruynooghe)
Date: Tue, 1 Mar 2011 08:50:11 +0000
Subject: [py-dev] pytest for python 3 ?
In-Reply-To: <AANLkTinqsZL-=+MPcR5C4H-r5dBBfvagP0f0bSimGxjq@mail.gmail.com>
References: <AANLkTinqsZL-=+MPcR5C4H-r5dBBfvagP0f0bSimGxjq@mail.gmail.com>
Message-ID: <AANLkTikGqyUiq9GXnsKB0UMTS6weW8vcYmSE+ODv05_m@mail.gmail.com>

On 1 March 2011 01:57, James Mills <prologic at shortcircuit.net.au> wrote:
> Has this happened yet ?

Yes that works.  The description over at the cheeseshop says
"Interpreters: Python versions 2.4 through to 3.2, Jython 2.5.1 and
PyPy"


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org


From memedough at gmail.com  Sun Mar  6 13:42:20 2011
From: memedough at gmail.com (meme dough)
Date: Sun, 6 Mar 2011 23:42:20 +1100
Subject: [py-dev] New issue 1 in pytest-cov: Allow branch coverage
In-Reply-To: <4D6F95D6.9000605@nedbatchelder.com>
References: <d98e9c9a5ea00f9fc32ef13b64e78e6d@bitbucket.org>
	<AANLkTinvyetm+qXeJ2P=NpUqyf0ZsNpxf2wPz1T62zU=@mail.gmail.com>
	<4D6F95D6.9000605@nedbatchelder.com>
Message-ID: <AANLkTinY-wS7pLkWS6UnhCGUwRZ+BZXdo5ha1VqLN4ke@mail.gmail.com>

Hi,

Thanks Ned.  That's pretty much what I've been thinking.  As much as
I'd like to have branch coverage available easily it does raise all
the pass through issues.

The command line options I think are very minimal and nice.  --cov
set's a source tree root, can have any number.  --cov-report produces
a report, can have any number.  --cov-config set's the config file for
full control over all coverage features.

:)

On 4 March 2011 00:21, Ned Batchelder <ned at nedbatchelder.com> wrote:
> One of the reasons I implemented the config file support was specifically to
> solve the "pass-through UI" problem with test runners. ?Adding a feature to
> coverage.py was useless unless the test runner or plugin also supported the
> switch. ?I'm all in favor of getting off that treadmill by encouraging
> people to use the config file to control coverage.py. ?I'd use a message of,
> "the command line is for simple use, once you are using coverage.py in a
> more sophisticated way, create a config file."
>
> --Ned.
>
> On 2/26/2011 2:42 AM, meme dough wrote:
>>
>> Hi,
>>
>> I'm cross posting to pytest and coverage to discuss.
>>
>> At the start pytest-cov had command line options to control all of
>> coverage.
>>
>> Talk on pytest mailing list there was concern about so many options
>> and that it should have minimal options. ?That way py.test --help
>> output is shorter and more useful as a single plugin doesn't output
>> huge amounts of text hiding the other plugin help text. ?Also it could
>> be fully controlled through coverage config file.
>>
>> We could add a small number of options in but I'd like to get others
>> opinions first. ?Plus how do we decide which ones should be in or not.
>> ?If we add branch then maybe someone else wants timid or cover pylib.
>>
>> Another thing to consider is should coverage have env vars for all of
>> it's options. ?That way it could be controlled from the command line
>> but independent of the test runner (pytest, nose, unittest2) and so it
>> would work with all of them. Something like:
>>
>> {{{
>> COVERAGE_BRANCH=1 py.test --cov-report term-missing --cov myproj tests/
>> }}}
>>
>> Or the following as coverage already reads COVERAGE_OPTIONS to get timid
>> option:
>>
>> {{{
>> COVERAGE_OPTIONS="--branch" py.test --cov-report term-missing --cov
>> myproj tests/
>> }}}
>>
>> These days I quite like the minimal options with full control through
>> config file. ?But I do see that it is nice to control through the
>> command line which is why I had full control of every option at the
>> very beginning (in fact I had command line option, env var, then
>> config file for each option).
>>
>> Any ideas on leaving it as it is? Adding small number of options to
>> pytest-cov? Adding env vars to coverage to control independently so
>> works will all test runners?
>>
>> :)
>>
>> On 26 February 2011 08:36, Bitbucket<issues-noreply at bitbucket.org> ?wrote:
>>>
>>> --- you can reply above this line ---
>>>
>>> New issue 1: Allow branch coverage
>>> https://bitbucket.org/memedough/pytest-cov/issue/1/allow-branch-coverage
>>>
>>> jimslack on Fri, 25 Feb 2011 22:36:33 +0100:
>>>
>>> Description:
>>> ?I know it is possible to activate branch coverage using a config file,
>>> but I'd like to use branch coverage with a command-line option. ?Perhaps
>>> something like:
>>>
>>> {{{
>>> py.test --cov-report term-missing --cov --branch myproj tests/
>>> }}}
>>>
>>> or
>>>
>>> {{{
>>> py.test --cov-report term-missing --covbranch myproj tests/
>>> }}}
>>>
>>> Another possibility would be to make branch coverage the default.
>>>
>>> Thanks.
>>>
>>>
>>>
>>>
>>> --
>>>
>>> This is an issue notification from bitbucket.org. You are receiving
>>> this either because you are the owner of the issue, or you are
>>> following the issue.
>>>
>


From holger at merlinux.eu  Wed Mar  9 14:03:42 2011
From: holger at merlinux.eu (holger krekel)
Date: Wed, 9 Mar 2011 13:03:42 +0000
Subject: [py-dev] pytest-2.0.2: bug fixes, improved xfail/skip expressions,
	speedups
Message-ID: <20110309130342.GK16231@merlinux.eu>


Welcome to pytest-2.0.2, a maintenance and bug fix release.

py.test is a mature testing tool for Python, supporting CPython 2.4-3.2,
Jython and latest PyPy interpreters.  See the extensive docs with tested
examples here:

    http://pytest.org/

If you want to install or upgrade pytest, just type one of::

    pip install -U pytest # or
    easy_install -U pytest

Many thanks to all issue reporters and people asking questions
or complaining, particularly Jurko for his insistence,
Laura, Victor and Brianna for helping with improving
and Ronny for his general advise.

best,
holger krekel

Changes between 2.0.1 and 2.0.2
----------------------------------------------

- tackle issue32 - speed up test runs of very quick test functions
  by reducing the relative overhead

- fix issue30 - extended xfail/skipif handling and improved reporting.
  If you have a syntax error in your skip/xfail
  expressions you now get nice error reports.

  Also you can now access module globals from xfail/skipif
  expressions so that this for example works now::
 
    import pytest
    import mymodule
    @pytest.mark.skipif("mymodule.__version__[0] == "1")
    def test_function():
        pass

  This will not run the test function if the module's version string 
  does not start with a "1".  Note that specifying a string instead
  of a boolean expressions allows py.test to report meaningful information 
  when summarizing a test run as to what conditions lead to skipping 
  (or xfail-ing) tests.

- fix issue28 - setup_method and pytest_generate_tests work together
  The setup_method fixture method now gets called also for
  test function invocations generated from the pytest_generate_tests
  hook.

- fix issue27 - collectonly and keyword-selection (-k) now work together
  Also, if you do "py.test --collectonly -q" you now get a flat list
  of test ids that you can use to paste to the py.test commandline
  in order to execute a particular test.

- fix issue25 avoid reported problems with --pdb and python3.2/encodings output

- fix issue23 - tmpdir argument now works on Python3.2 and WindowsXP
  Starting with Python3.2 os.symlink may be supported. By requiring
  a newer py lib version the py.path.local() implementation acknowledges
  this.

- fixed typos in the docs (thanks Victor Garcia, Brianna Laugher) and particular
  thanks to Laura Creighton who also revieved parts of the documentation.

- fix slighly wrong output of verbose progress reporting for classes 
  (thanks Amaury)

- more precise (avoiding of) deprecation warnings for node.Class|Function accesses

- avoid std unittest assertion helper code in tracebacks (thanks Ronny)


From cklinger at novareto.de  Mon Mar 14 00:07:35 2011
From: cklinger at novareto.de (Christian Klinger)
Date: Mon, 14 Mar 2011 00:07:35 +0100
Subject: [py-dev] py.test doctest and hooks...
In-Reply-To: <20110119181139.GA15381@trillke.net>
References: <4D36EA7A.5020801@novareto.de> <20110119181139.GA15381@trillke.net>
Message-ID: <AANLkTimWn71uZv22UbuDM6zMoN_XDsrQrCLh-qsN_mHJ@mail.gmail.com>

Hi Holger,

sorry for answering so late...

we are using doctest files like ('test_*.txt').
It seems that none of the hooks works.

Can you confirm this impression.

THX
Christian

2011/1/19 holger krekel <holger at merlinux.eu>

> Hi Christian,
>
> On Wed, Jan 19, 2011 at 14:43 +0100, Christian Klinger wrote:
> > Hello,
> >
> > are there any hooks in pytest which are executed before doctests are
> > executed?
> > I try to setup some stuff, which i want to hide in the doctest.
>
> doctest modules or doctest files ("test_*.txt")?
>
> > I already tried some "normal" pytest hooks ala:
> >
> > - pytest_runtest_call(item):- pytest_runtest_setup(item): But none of
> > them seems to be executed on doctest execution. Any ideas? Christian
>
> Works for me.  Just tried a conftest.py like this:
>
>    import pytest
>
>    def pytest_runtest_setup(item):
>        pytest.set_trace()
>
> and in the same dir a doc.txt like this:
>
>    >>> 3
>    4
>
> and then run "py.test -s" which invokes the setup hook
> using pytest-2.0.0 for me.
>
> Not sure if this is sufficient for your setup purposes, though.
> Do you want to inject custom objects into the namespace
> for your doctests by chance?
>
> If so you might want to look into
>
>    https://bitbucket.org/hpk42/pytest/src/88c0e00cfe8b/_pytest/doctest.py
>
> and enhance it to suit your purposes (pass "extraglobs") and then submit
> back
> the mods including a test.
>
> On a more general note, it actually makes might make
> sense to port the way nose is doing it, see here:
>
>
> http://packages.python.org/nose/doc_tests/test_doctest_fixtures/doctest_fixtures.html
>
> cheers,
> holger
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20110314/e6e74148/attachment.html>

From prologic at shortcircuit.net.au  Mon Mar 14 00:22:11 2011
From: prologic at shortcircuit.net.au (James Mills)
Date: Mon, 14 Mar 2011 09:22:11 +1000
Subject: [py-dev] Testing web services
Message-ID: <AANLkTimMUn5gO+7UF5dzvyhHd0z8M3vyQf+dZtZtun0b@mail.gmail.com>

Hello all,

I'm thinking about how to go about testing
web services in a new services-oriented
application framework we're working on.

Are there any assisting libraries, tools,
hooks or plugins to make this job a litt
easier with pytest ?

Right now, I'm taking the naive approach which is
to simply start the application in a ``setup_module(...)``,
perform the necessary tests, then shut the application
down in a ``teardown_module(...)``. I was simply going
to be using ``urllib`` to perform requests against it
and assert it's results.

Maybe there's a more sophisticated way ?

In the end, we will likely setup and use selenium
to do more rigorous system testing, but right now
I'm looking for more service-level testing at the
data/logic layers of our application.

cheers
James

-- 
-- James Mills
--
-- "Problems are solved by method"


From prologic at shortcircuit.net.au  Mon Mar 14 07:57:07 2011
From: prologic at shortcircuit.net.au (James Mills)
Date: Mon, 14 Mar 2011 16:57:07 +1000
Subject: [py-dev] pytest_cov: start session before test improts
Message-ID: <AANLkTiks_q-81HiDUCb3AW100LAFtBSzn44Eac6i0hkB@mail.gmail.com>

Helo all,

>From the FAQ of the coverage module (1):

"""
Q: Why do the bodies of functions (or classes) show as executed, but
the def lines do not?

This happens because coverage is started after the functions are
defined. The definition lines are executed without coverage
measurement, then coverage is started, then the function is called.
This means the body is measured, but the definition of the function
itself is not.

To fix this, start coverage earlier. If you use the command line to
run your program with coverage, then your entire program will be
monitored. If you are using the API, you need to call coverage.start()
before importing the modules that define your functions.
"""

I'm not exactly sure (after some playing with pytest_cov plugin) how
to do this...

I'll take another look tonight if I can, but appreciate someone more experienced
with pytest internals to take a look :)

cheers
James

1. http://nedbatchelder.com/code/coverage/faq.html#faq

-- 
-- James Mills
--
-- "Problems are solved by method"


From holger at merlinux.eu  Mon Mar 14 13:29:49 2011
From: holger at merlinux.eu (holger krekel)
Date: Mon, 14 Mar 2011 12:29:49 +0000
Subject: [py-dev] py.test doctest and hooks...
In-Reply-To: <AANLkTimWn71uZv22UbuDM6zMoN_XDsrQrCLh-qsN_mHJ@mail.gmail.com>
References: <4D36EA7A.5020801@novareto.de> <20110119181139.GA15381@trillke.net>
	<AANLkTimWn71uZv22UbuDM6zMoN_XDsrQrCLh-qsN_mHJ@mail.gmail.com>
Message-ID: <20110314122949.GV16231@merlinux.eu>

Hi Christian, 

On Mon, Mar 14, 2011 at 00:07 +0100, Christian Klinger wrote:
> Hi Holger,
> 
> sorry for answering so late...
> 
> we are using doctest files like ('test_*.txt').
> It seems that none of the hooks works.
> 
> Can you confirm this impression.

at this point i'd like to have a minimal example, i.e.
some files and what you type and what you think should
happen and doesn't.  I then will look into helping to
fix it.

holger

> THX
> Christian
> 
> 2011/1/19 holger krekel <holger at merlinux.eu>
> 
> > Hi Christian,
> >
> > On Wed, Jan 19, 2011 at 14:43 +0100, Christian Klinger wrote:
> > > Hello,
> > >
> > > are there any hooks in pytest which are executed before doctests are
> > > executed?
> > > I try to setup some stuff, which i want to hide in the doctest.
> >
> > doctest modules or doctest files ("test_*.txt")?
> >
> > > I already tried some "normal" pytest hooks ala:
> > >
> > > - pytest_runtest_call(item):- pytest_runtest_setup(item): But none of
> > > them seems to be executed on doctest execution. Any ideas? Christian
> >
> > Works for me.  Just tried a conftest.py like this:
> >
> >    import pytest
> >
> >    def pytest_runtest_setup(item):
> >        pytest.set_trace()
> >
> > and in the same dir a doc.txt like this:
> >
> >    >>> 3
> >    4
> >
> > and then run "py.test -s" which invokes the setup hook
> > using pytest-2.0.0 for me.
> >
> > Not sure if this is sufficient for your setup purposes, though.
> > Do you want to inject custom objects into the namespace
> > for your doctests by chance?
> >
> > If so you might want to look into
> >
> >    https://bitbucket.org/hpk42/pytest/src/88c0e00cfe8b/_pytest/doctest.py
> >
> > and enhance it to suit your purposes (pass "extraglobs") and then submit
> > back
> > the mods including a test.
> >
> > On a more general note, it actually makes might make
> > sense to port the way nose is doing it, see here:
> >
> >
> > http://packages.python.org/nose/doc_tests/test_doctest_fixtures/doctest_fixtures.html
> >
> > cheers,
> > holger
> >


From prologic at shortcircuit.net.au  Mon Mar 14 13:42:38 2011
From: prologic at shortcircuit.net.au (James Mills)
Date: Mon, 14 Mar 2011 22:42:38 +1000
Subject: [py-dev] pytest_cov: start session before test improts
In-Reply-To: <AANLkTimXSrdJamZ4hm8tWYZaTvdg6ojgFySw2iBLZdLg@mail.gmail.com>
References: <AANLkTiks_q-81HiDUCb3AW100LAFtBSzn44Eac6i0hkB@mail.gmail.com>
	<AANLkTimXSrdJamZ4hm8tWYZaTvdg6ojgFySw2iBLZdLg@mail.gmail.com>
Message-ID: <AANLkTik==gcFbQM6tcioKSW5q9jzFxAjYwTWj75wLt7o@mail.gmail.com>

On Mon, Mar 14, 2011 at 10:07 PM, meme dough <memedough at gmail.com> wrote:
> I just tried with one of my projects and it's got the def lines and
> all the imports as covered.
>
> Do you have a small example so I can see? ?The command you're using
> and the the versions would be good too. ?Maybe drop at
> http://paste.pocoo.org/

After some testing I reproduced the problem.

The behavior I'm seeing only occurs with Python 3.2

Would this be a problem with pytest, puytest_cov or coverage wrt Python 3 ?

Bed time :)

cheers
James

-- 
-- James Mills
--
-- "Problems are solved by method"


From prologic at shortcircuit.net.au  Mon Mar 14 13:43:39 2011
From: prologic at shortcircuit.net.au (James Mills)
Date: Mon, 14 Mar 2011 22:43:39 +1000
Subject: [py-dev] Testing web services
In-Reply-To: <4D7DC651.9070808@gmx.net>
References: <AANLkTimMUn5gO+7UF5dzvyhHd0z8M3vyQf+dZtZtun0b@mail.gmail.com>
	<4D7DC651.9070808@gmx.net>
Message-ID: <AANLkTinpMcF76jYgE2GVpRFDoA9K=hcpn7eJzvKYRiCB@mail.gmail.com>

On Mon, Mar 14, 2011 at 5:40 PM, Frederik Dohr <fdg001 at gmx.net> wrote:
> FWIW, TiddlyWeb's HTTP API is being tested using httplib2, with tests
> described concisely in YAML:
> https://github.com/tiddlyweb/tiddlyweb/blob/master/test/httptest.yaml
> https://github.com/tiddlyweb/tiddlyweb/blob/master/test/test_web_http_api.py
> https://github.com/tiddlyweb/tiddlyweb/blob/master/test/http_runner.py

Thank you! Judging from the YAML declarations this looks really good.

cheers
James

-- 
-- James Mills
--
-- "Problems are solved by method"


From prologic at shortcircuit.net.au  Mon Mar 14 13:43:39 2011
From: prologic at shortcircuit.net.au (James Mills)
Date: Mon, 14 Mar 2011 22:43:39 +1000
Subject: [py-dev] Testing web services
In-Reply-To: <4D7DC651.9070808@gmx.net>
References: <AANLkTimMUn5gO+7UF5dzvyhHd0z8M3vyQf+dZtZtun0b@mail.gmail.com>
	<4D7DC651.9070808@gmx.net>
Message-ID: <AANLkTinpMcF76jYgE2GVpRFDoA9K=hcpn7eJzvKYRiCB@mail.gmail.com>

On Mon, Mar 14, 2011 at 5:40 PM, Frederik Dohr <fdg001 at gmx.net> wrote:
> FWIW, TiddlyWeb's HTTP API is being tested using httplib2, with tests
> described concisely in YAML:
> https://github.com/tiddlyweb/tiddlyweb/blob/master/test/httptest.yaml
> https://github.com/tiddlyweb/tiddlyweb/blob/master/test/test_web_http_api.py
> https://github.com/tiddlyweb/tiddlyweb/blob/master/test/http_runner.py

Thank you! Judging from the YAML declarations this looks really good.

cheers
James

-- 
-- James Mills
--
-- "Problems are solved by method"


From fdg001 at gmx.net  Mon Mar 14 14:09:30 2011
From: fdg001 at gmx.net (Frederik Dohr)
Date: Mon, 14 Mar 2011 14:09:30 +0100
Subject: [py-dev] Testing web services
In-Reply-To: <AANLkTimMUn5gO+7UF5dzvyhHd0z8M3vyQf+dZtZtun0b@mail.gmail.com>
References: <AANLkTimMUn5gO+7UF5dzvyhHd0z8M3vyQf+dZtZtun0b@mail.gmail.com>
Message-ID: <4D7E138A.9030202@gmx.net>

> I'm thinking about how to go about testing web services in a new
> services-oriented application framework we're working on.

FWIW, TiddlyWeb's HTTP API is being tested using httplib2, with tests
described concisely in YAML:
https://github.com/tiddlyweb/tiddlyweb/blob/master/test/httptest.yaml
https://github.com/tiddlyweb/tiddlyweb/blob/master/test/test_web_http_api.py
https://github.com/tiddlyweb/tiddlyweb/blob/master/test/http_runner.py


-- F.


From holger at merlinux.eu  Mon Mar 14 14:42:39 2011
From: holger at merlinux.eu (holger krekel)
Date: Mon, 14 Mar 2011 13:42:39 +0000
Subject: [py-dev] Testing web services
In-Reply-To: <4D7E138A.9030202@gmx.net>
References: <AANLkTimMUn5gO+7UF5dzvyhHd0z8M3vyQf+dZtZtun0b@mail.gmail.com>
	<4D7E138A.9030202@gmx.net>
Message-ID: <20110314134239.GX16231@merlinux.eu>

On Mon, Mar 14, 2011 at 14:09 +0100, Frederik Dohr wrote:
> > I'm thinking about how to go about testing web services in a new
> > services-oriented application framework we're working on.
> 
> FWIW, TiddlyWeb's HTTP API is being tested using httplib2, with tests
> described concisely in YAML:
> https://github.com/tiddlyweb/tiddlyweb/blob/master/test/httptest.yaml
> https://github.com/tiddlyweb/tiddlyweb/blob/master/test/test_web_http_api.py
> https://github.com/tiddlyweb/tiddlyweb/blob/master/test/http_runner.py

interesting.
Do you run all the yaml-defined tests from the setup_module?

Out of curiostiy, have you looked 
into http://pytest.org/example/nonpython.html ?

It presents an example of how to have YAML defined tests integrate
into a test run (and seeing a dot for each passing test).

cheers,
holger

> 
> -- F.
> _______________________________________________
> py-dev mailing list
> py-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/py-dev
> 


From fdg001 at gmx.net  Mon Mar 14 15:48:24 2011
From: fdg001 at gmx.net (Frederik Dohr)
Date: Mon, 14 Mar 2011 15:48:24 +0100
Subject: [py-dev] Testing web services
In-Reply-To: <20110314134239.GX16231@merlinux.eu>
References: <AANLkTimMUn5gO+7UF5dzvyhHd0z8M3vyQf+dZtZtun0b@mail.gmail.com>
	<4D7E138A.9030202@gmx.net> <20110314134239.GX16231@merlinux.eu>
Message-ID: <4D7E2AB8.4040703@gmx.net>

I should clarify: I'm not the original author of that YAML setup; Chris
Dent is.

> Do you run all the yaml-defined tests from the setup_module?

No; the test_the_TESTS function is imported into the test module, which
then yields the individual tests.

> Out of curiostiy, have you looked 
> into http://pytest.org/example/nonpython.html ?

I must admit no; essentially, this current setup was conceived as the
simplest thing that could possibly work. Generally, we're striving to
stick to the very basics - which, admittedly, sometimes leads to
reinventing the wheel.


-- F.


From prologic at shortcircuit.net.au  Mon Mar 14 22:19:46 2011
From: prologic at shortcircuit.net.au (James Mills)
Date: Tue, 15 Mar 2011 07:19:46 +1000
Subject: [py-dev] Testing web services
In-Reply-To: <20110314134239.GX16231@merlinux.eu>
References: <AANLkTimMUn5gO+7UF5dzvyhHd0z8M3vyQf+dZtZtun0b@mail.gmail.com>
	<4D7E138A.9030202@gmx.net> <20110314134239.GX16231@merlinux.eu>
Message-ID: <AANLkTik2hUREfyh9gFj3T188A7-k8eOm=PdcAUe1wJX_@mail.gmail.com>

On Mon, Mar 14, 2011 at 11:42 PM, holger krekel <holger at merlinux.eu> wrote:
> Out of curiostiy, have you looked
> into http://pytest.org/example/nonpython.html ?
>
> It presents an example of how to have YAML defined tests integrate
> into a test run (and seeing a dot for each passing test).

Thanks Holger :) This should prove useful.
Exactly what I want ... Test some web framework or web services application
and have pytest report on it and get some coverage data too :)

cheers
James

-- 
-- James Mills
--
-- "Problems are solved by method"


From prologic at shortcircuit.net.au  Tue Mar 15 00:28:06 2011
From: prologic at shortcircuit.net.au (James Mills)
Date: Tue, 15 Mar 2011 09:28:06 +1000
Subject: [py-dev] Testing web services
In-Reply-To: <4D7DC651.9070808@gmx.net>
References: <AANLkTimMUn5gO+7UF5dzvyhHd0z8M3vyQf+dZtZtun0b@mail.gmail.com>
	<4D7DC651.9070808@gmx.net>
Message-ID: <AANLkTin_5FzoNbLnWnf-SJ+t9WcYmnW0uTf=ee+AqE20@mail.gmail.com>

On Mon, Mar 14, 2011 at 5:40 PM, Frederik Dohr <fdg001 at gmx.net> wrote:
> FWIW, TiddlyWeb's HTTP API is being tested using httplib2, with tests
> described concisely in YAML:
> https://github.com/tiddlyweb/tiddlyweb/blob/master/test/httptest.yaml
> https://github.com/tiddlyweb/tiddlyweb/blob/master/test/test_web_http_api.py
> https://github.com/tiddlyweb/tiddlyweb/blob/master/test/http_runner.py

I wonder how hard it would be to make this work
with pytest and the example of running non-python test
that Holger presented here: http://pytest.org/example/nonpython.html

I'll have a go at it today...

cheers
James

-- 
-- James Mills
--
-- "Problems are solved by method"


From prologic at shortcircuit.net.au  Tue Mar 15 02:14:38 2011
From: prologic at shortcircuit.net.au (James Mills)
Date: Tue, 15 Mar 2011 11:14:38 +1000
Subject: [py-dev] pytest_cov: start session before test improts
In-Reply-To: <AANLkTik==gcFbQM6tcioKSW5q9jzFxAjYwTWj75wLt7o@mail.gmail.com>
References: <AANLkTiks_q-81HiDUCb3AW100LAFtBSzn44Eac6i0hkB@mail.gmail.com>
	<AANLkTimXSrdJamZ4hm8tWYZaTvdg6ojgFySw2iBLZdLg@mail.gmail.com>
	<AANLkTik==gcFbQM6tcioKSW5q9jzFxAjYwTWj75wLt7o@mail.gmail.com>
Message-ID: <AANLkTinQhKjpzgyq3zSY-MznwgnU46-6uLxq7zOKyody@mail.gmail.com>

On Mon, Mar 14, 2011 at 10:42 PM, James Mills
<prologic at shortcircuit.net.au> wrote:
> After some testing I reproduced the problem.
>
> The behavior I'm seeing only occurs with Python 3.2
>
> Would this be a problem with pytest, puytest_cov or coverage wrt Python 3 ?

Okay. Lesson learned.

If you import from your package/modules that you're testing
in conftest.py (say you want to provide some common funcargs)
then you'll see this behavior.

If however you import within the funcarg function itself
then it's all "good" :)

Tip: Don't do top-level imports of your package/modules under
test in conftest.py

cheers
James

-- 
-- James Mills
--
-- "Problems are solved by method"


From prologic at shortcircuit.net.au  Tue Mar 15 02:59:54 2011
From: prologic at shortcircuit.net.au (James Mills)
Date: Tue, 15 Mar 2011 11:59:54 +1000
Subject: [py-dev] Testing web services
In-Reply-To: <AANLkTin_5FzoNbLnWnf-SJ+t9WcYmnW0uTf=ee+AqE20@mail.gmail.com>
References: <AANLkTimMUn5gO+7UF5dzvyhHd0z8M3vyQf+dZtZtun0b@mail.gmail.com>
	<4D7DC651.9070808@gmx.net>
	<AANLkTin_5FzoNbLnWnf-SJ+t9WcYmnW0uTf=ee+AqE20@mail.gmail.com>
Message-ID: <AANLkTi=WaQhErCq1PpOPHqjQksMA7e+eGb7+YEj8jXv_@mail.gmail.com>

On Tue, Mar 15, 2011 at 9:28 AM, James Mills
<prologic at shortcircuit.net.au> wrote:
> I wonder how hard it would be to make this work
> with pytest and the example of running non-python test
> that Holger presented here: http://pytest.org/example/nonpython.html
>
> I'll have a go at it today...

I believe Ali's pytest- (1) plugin is a great start to
integrating declarative web-oriented tests (in yaml)
into pytest.

I'd like to see two improvements though:

1) Remove dependency on werkzeug
2) Add support for more flexible setups (app_factory, host/port, etc)

cheers
James

1. http://pypi.python.org/pypi/pytest-yamlwsgi/0.6

-- 
-- James Mills
--
-- "Problems are solved by method"


From prologic at shortcircuit.net.au  Tue Mar 15 03:00:34 2011
From: prologic at shortcircuit.net.au (James Mills)
Date: Tue, 15 Mar 2011 12:00:34 +1000
Subject: [py-dev] Testing web services
In-Reply-To: <AANLkTi=WaQhErCq1PpOPHqjQksMA7e+eGb7+YEj8jXv_@mail.gmail.com>
References: <AANLkTimMUn5gO+7UF5dzvyhHd0z8M3vyQf+dZtZtun0b@mail.gmail.com>
	<4D7DC651.9070808@gmx.net>
	<AANLkTin_5FzoNbLnWnf-SJ+t9WcYmnW0uTf=ee+AqE20@mail.gmail.com>
	<AANLkTi=WaQhErCq1PpOPHqjQksMA7e+eGb7+YEj8jXv_@mail.gmail.com>
Message-ID: <AANLkTimkaX4YkWrBF=Sm-jOesHo9LMPXiMsLEhLVmd1q@mail.gmail.com>

On Tue, Mar 15, 2011 at 11:59 AM, James Mills
<prologic at shortcircuit.net.au> wrote:
> I believe Ali's pytest- (1) plugin is a great start to
> integrating declarative web-oriented tests (in yaml)
> into pytest.

pytest-yamlwsgi/*

-- 
-- James Mills
--
-- "Problems are solved by method"


From memedough at gmail.com  Sun Mar 20 12:00:59 2011
From: memedough at gmail.com (meme dough)
Date: Sun, 20 Mar 2011 22:00:59 +1100
Subject: [py-dev] latest pytest questions for pytest-cov
Message-ID: <AANLkTinG=NKTGKY+Cv+ixRr8O+2nAnTW0_Cx5fOmi+7r@mail.gmail.com>

Hi,

Currently looking at fixing pytest-cov to work with latest pytest.  I
see two issues so far.

1. How to determine if non distributed, or distributed master, or
distributed slave.

This used to look at the session class Session = Central (non
distributed), DSession = distributed master, and SlaveSession =
distributed slave.

So now I'm thinking look at
session.config.pluginmanager.hasplugin('dsession') which means
distributed master.  For distributed slave check if config.slaveinput
is there which means distributed slave.  If none of those then must be
non distributed.

Is there a better way to determine at pytest_sessionstart if the
process is central (non distributed), distributed master or
distributed slave?

2. Need to know topdir.

This used to use config.topdir to know what the root dir is.  This is
needed since distributed testing may be co-located or not.  If not
then have to rewrite all source paths back to what the master is set
to so that all coverage files combine correctly even if have many
separate tx envs on diff machines on diff paths.

Has this moved somewhere?  I see config.fspath, is that it?  I look
more I just got to this bit.

:)


From holger at merlinux.eu  Sun Mar 20 12:21:20 2011
From: holger at merlinux.eu (holger krekel)
Date: Sun, 20 Mar 2011 11:21:20 +0000
Subject: [py-dev] latest pytest questions for pytest-cov
In-Reply-To: <AANLkTinG=NKTGKY+Cv+ixRr8O+2nAnTW0_Cx5fOmi+7r@mail.gmail.com>
References: <AANLkTinG=NKTGKY+Cv+ixRr8O+2nAnTW0_Cx5fOmi+7r@mail.gmail.com>
Message-ID: <20110320112120.GL16231@merlinux.eu>

Hi Meme,

On Sun, Mar 20, 2011 at 22:00 +1100, meme dough wrote:
> Hi,
> 
> Currently looking at fixing pytest-cov to work with latest pytest.  I
> see two issues so far.
> 
> 1. How to determine if non distributed, or distributed master, or
> distributed slave.
> 
> This used to look at the session class Session = Central (non
> distributed), DSession = distributed master, and SlaveSession =
> distributed slave.
> 
> So now I'm thinking look at
> session.config.pluginmanager.hasplugin('dsession') which means
> distributed master.  For distributed slave check if config.slaveinput
> is there which means distributed slave.  If none of those then must be
> non distributed.

I guess that's ok.  There is no "official" way at the moment.

> Is there a better way to determine at pytest_sessionstart if the
> process is central (non distributed), distributed master or
> distributed slave?
> 
> 2. Need to know topdir.
> 
> This used to use config.topdir to know what the root dir is.  This is
> needed since distributed testing may be co-located or not.  If not
> then have to rewrite all source paths back to what the master is set
> to so that all coverage files combine correctly even if have many
> separate tx envs on diff machines on diff paths.
> 
> Has this moved somewhere?  I see config.fspath, is that it?  I look
> more I just got to this bit.

No, there now is a "looponfailroots" (see --help) and distributed
testing uses "rsyncroots". The problem is that "topdir" was a bit
of an artifical concept.

Maybe it helps, that item.nodeid contains a "::" separated test id where
the first part is a path relative to the current directory and
normalized to use "/" as separator.

best,
holger


From memedough at gmail.com  Sun Mar 27 13:28:42 2011
From: memedough at gmail.com (meme dough)
Date: Sun, 27 Mar 2011 22:28:42 +1100
Subject: [py-dev] latest pytest questions for pytest-cov
In-Reply-To: <20110320112120.GL16231@merlinux.eu>
References: <AANLkTinG=NKTGKY+Cv+ixRr8O+2nAnTW0_Cx5fOmi+7r@mail.gmail.com>
	<20110320112120.GL16231@merlinux.eu>
Message-ID: <AANLkTimr9-AwUvngsh6ofWTq6btiKzH6+9YpgYQ_1aRg@mail.gmail.com>

Hi,

Now up and working with the latest pytest (think only distributed
testing was an issue).  It is still backward compatible with older
versions of pytest (before the split).

Funcarg is at module level so visible with py.test --funcargs.

pth file more robust since it doesn't complain if cov-core got deleted
and pth file left.

More doc for coverage config file since couple of people have asked questions.

I moved the doc to the docstring in the module.  I see that the
website just points to the external plugins at pypi, so that means the
doc can't get out of sync right?

:)

On 20 March 2011 22:21, holger krekel <holger at merlinux.eu> wrote:
> Hi Meme,
>
> On Sun, Mar 20, 2011 at 22:00 +1100, meme dough wrote:
>> Hi,
>>
>> Currently looking at fixing pytest-cov to work with latest pytest. ?I
>> see two issues so far.
>>
>> 1. How to determine if non distributed, or distributed master, or
>> distributed slave.
>>
>> This used to look at the session class Session = Central (non
>> distributed), DSession = distributed master, and SlaveSession =
>> distributed slave.
>>
>> So now I'm thinking look at
>> session.config.pluginmanager.hasplugin('dsession') which means
>> distributed master. ?For distributed slave check if config.slaveinput
>> is there which means distributed slave. ?If none of those then must be
>> non distributed.
>
> I guess that's ok. ?There is no "official" way at the moment.
>
>> Is there a better way to determine at pytest_sessionstart if the
>> process is central (non distributed), distributed master or
>> distributed slave?
>>
>> 2. Need to know topdir.
>>
>> This used to use config.topdir to know what the root dir is. ?This is
>> needed since distributed testing may be co-located or not. ?If not
>> then have to rewrite all source paths back to what the master is set
>> to so that all coverage files combine correctly even if have many
>> separate tx envs on diff machines on diff paths.
>>
>> Has this moved somewhere? ?I see config.fspath, is that it? ?I look
>> more I just got to this bit.
>
> No, there now is a "looponfailroots" (see --help) and distributed
> testing uses "rsyncroots". The problem is that "topdir" was a bit
> of an artifical concept.
>
> Maybe it helps, that item.nodeid contains a "::" separated test id where
> the first part is a path relative to the current directory and
> normalized to use "/" as separator.
>
> best,
> holger
>