From flub at devork.be  Fri Oct  1 01:23:20 2010
From: flub at devork.be (Floris Bruynooghe)
Date: Fri, 1 Oct 2010 00:23:20 +0100
Subject: [py-dev] controlling assertions and difflib
In-Reply-To: <20100929132818.GK20695@trillke.net>
References: <AANLkTimf7aY-BczSSVvfKFcNvD5BTz6bpE67OEsWyXjM@mail.gmail.com>
	<20100929132818.GK20695@trillke.net>
Message-ID: <AANLkTinY-2_d5+RqhEJ0dGXmY=PFJHwc_opQi=P-2OL5@mail.gmail.com>

Hello

On 29 September 2010 14:28, holger krekel <holger at merlinux.eu> wrote:
> On Wed, Sep 29, 2010 at 00:12 +0100, Floris Bruynooghe wrote:
>> ______________________________ test_hang_in_diff _______________________________
>> /tmp/sandbox/test_bugs.py:4: in test_hang_in_diff
>> > ? ? ? assert x == y
>> E ? ? ? assert '1\n1\n1\n1\n...n2\n2\n2\n2\n' == '1\n1\n1\n1\n...n2\n2\n2\n2\n'
>> E ? ? ? ? Skipping 1990 identical leading characters in diff
>> E ? ? ? ? Skipping 1991 identical trailing characters in diff
>> E ? ? ? ? ? 1
>> E ? ? ? ? ? 1
>> E ? ? ? ? ? 1
>> E ? ? ? ? ? 1
>> E ? ? ? ? ? 1
>> E ? ? ? ? - abc2
>> E ? ? ? ? + def2
>> E ? ? ? ? ? 2
>> E ? ? ? ? ? 2
>> E ? ? ? ? ? 2
>> E ? ? ? ? ? 2
>
>
> looks good to me!

I've committed a cleaned-up version of this now.


>> This whole issue made me wonder if it should be possible to disable
>> the pytest_assert_binrepr hook.
>
> We could do a general "--assertmode=(choice)" with e.g.:
>
> ? ?0: (no) don't do any assert reinterp (like --no-assert now)
> ? ?1: (basic) do basic reinterpreation
> ? ?2: (advanced) do basic reinterpretation + customizing hooks
>
> and default to 2.

Sounds like a good idea.  Do you think there's place for a
advanced-no-builtin-plugin option in here, so that the builtin hook
will not run any code?  Otherwise a bug in the builtin customisation
hook would completely stop this from being useful.


> I think there is an elegant solution. ?Instead of using "hook" in py/code/ we
> use a generic "customize" function that is to be called. ?In
> pytest_assertion.py's pytest_configure we write that function
> such that it calls
>
> ? ?pytest_assert_binrepr(config, ...)
>
> hope that makes sense. If in doubt leave it to me and just use py.test.config.

Do you mean something like the attached diff does?  In that case I'm
sort of tempted to remove the customise_assert() function I've added
there and just expose the DebugInterpreter so that you can modify the
attribute directly.  I also considered the
customise_assert(binrepr_hook, config) signature for this, not sure
which would be best.


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: config.diff
Type: text/x-patch
Size: 3108 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20101001/75ab3c7b/attachment.bin>

From holger at merlinux.eu  Sat Oct  2 19:33:43 2010
From: holger at merlinux.eu (holger krekel)
Date: Sat, 2 Oct 2010 19:33:43 +0200
Subject: [py-dev] controlling assertions and difflib
In-Reply-To: <AANLkTinY-2_d5+RqhEJ0dGXmY=PFJHwc_opQi=P-2OL5@mail.gmail.com>
References: <AANLkTimf7aY-BczSSVvfKFcNvD5BTz6bpE67OEsWyXjM@mail.gmail.com>
	<20100929132818.GK20695@trillke.net>
	<AANLkTinY-2_d5+RqhEJ0dGXmY=PFJHwc_opQi=P-2OL5@mail.gmail.com>
Message-ID: <20101002173342.GT20695@trillke.net>

Hey Floris,

On Fri, Oct 01, 2010 at 00:23 +0100, Floris Bruynooghe wrote:
> On 29 September 2010 14:28, holger krekel <holger at merlinux.eu> wrote:
> > On Wed, Sep 29, 2010 at 00:12 +0100, Floris Bruynooghe wrote:
> >> ______________________________ test_hang_in_diff _______________________________
> >> /tmp/sandbox/test_bugs.py:4: in test_hang_in_diff
> >> > ? ? ? assert x == y
> >> E ? ? ? assert '1\n1\n1\n1\n...n2\n2\n2\n2\n' == '1\n1\n1\n1\n...n2\n2\n2\n2\n'
> >> E ? ? ? ? Skipping 1990 identical leading characters in diff
> >> E ? ? ? ? Skipping 1991 identical trailing characters in diff
> >> E ? ? ? ? ? 1
> >> E ? ? ? ? ? 1
> >> E ? ? ? ? ? 1
> >> E ? ? ? ? ? 1
> >> E ? ? ? ? ? 1
> >> E ? ? ? ? - abc2
> >> E ? ? ? ? + def2
> >> E ? ? ? ? ? 2
> >> E ? ? ? ? ? 2
> >> E ? ? ? ? ? 2
> >> E ? ? ? ? ? 2
> >
> >
> > looks good to me!
> 
> I've committed a cleaned-up version of this now.

Great, thanks.  It's now merged to py-trunk!
 
> >> This whole issue made me wonder if it should be possible to disable
> >> the pytest_assert_binrepr hook.
> >
> > We could do a general "--assertmode=(choice)" with e.g.:
> >
> > ? ?0: (no) don't do any assert reinterp (like --no-assert now)
> > ? ?1: (basic) do basic reinterpreation
> > ? ?2: (advanced) do basic reinterpretation + customizing hooks
> >
> > and default to 2.
> 
> Sounds like a good idea.  Do you think there's place for a
> advanced-no-builtin-plugin option in here, so that the builtin hook
> will not run any code?  Otherwise a bug in the builtin customisation
> hook would completely stop this from being useful.

I am not so afraid but we can do it.  
 
> > I think there is an elegant solution. ?Instead of using "hook" in py/code/ we
> > use a generic "customize" function that is to be called. ?In
> > pytest_assertion.py's pytest_configure we write that function
> > such that it calls
> >
> > ? ?pytest_assert_binrepr(config, ...)
> >
> > hope that makes sense. If in doubt leave it to me and just use py.test.config.
> 
> Do you mean something like the attached diff does?  In that case I'm
> sort of tempted to remove the customise_assert() function I've added
> there and just expose the DebugInterpreter so that you can modify the
> attribute directly.  I also considered the
> customise_assert(binrepr_hook, config) signature for this, not sure
> which would be best.

I think i meant something different.  Please have a look into this 
changeset which i just added:

    http://bitbucket.org/hpk42/py-trunk/changeset/64962794201b

does this make sense to you? 

Also did a subsequent commit to rename to pytest_assertrepr_compare which
now can get a config object so we can now introduce variations controled 
by config values. 

best,
holger

> 
> 
> Regards
> Floris
> 
> 
> -- 
> Debian GNU/Linux -- The Power of Freedom
> www.debian.org | www.gnu.org | www.kernel.org

> diff --git a/py/__init__.py b/py/__init__.py
> --- a/py/__init__.py
> +++ b/py/__init__.py
> @@ -99,6 +99,7 @@ py.apipkg.initpkg(__name__, dict(
>          '_AssertionError'   : '._code.assertion:AssertionError',
>          '_reinterpret_old'  : '._code.assertion:reinterpret_old',
>          '_reinterpret'      : '._code.assertion:reinterpret',
> +        'customise_assert'  : '._code._assertionnew:customise_assert',
>      },
>  
>      # backports and additions of builtins
> diff --git a/py/_code/_assertionnew.py b/py/_code/_assertionnew.py
> --- a/py/_code/_assertionnew.py
> +++ b/py/_code/_assertionnew.py
> @@ -107,17 +107,21 @@ unary_map = {
>  }
>  
>  
> +def customise_assert(config):
> +    """Customise the DebugInterpreter with pytest config"""
> +    DebugInterpreter.pytest_config = config
> +
> +
>  class DebugInterpreter(ast.NodeVisitor):
>      """Interpret AST nodes to gleam useful debugging information.
>  
>      The _pytesthook attribute is used to detect if the py.test
>      pytest_assertion plugin is loaded and if so call it's hooks.
>      """
> +    pytest_config = None
>  
>      def __init__(self, frame):
>          self.frame = frame
> -        self._pytesthook = getattr(py.builtin.builtins.AssertionError,
> -                                   "_pytesthook")
>  
>      def generic_visit(self, node):
>          # Fallback when we don't have a special implementation.
> @@ -183,9 +191,10 @@ class DebugInterpreter(ast.NodeVisitor):
>              if not result:
>                  break
>              left_explanation, left_result = next_explanation, next_result
> -        if self._pytesthook:
> -            hook_result = self._pytesthook.pytest_assert_binrepr(
> -                op=op_symbol, left=left_result, right=next_result)
> +        if self.pytest_config:
> +            hook_result = self.pytest_config.hook.pytest_assert_binrepr(
> +                op=op_symbol, left=left_result, right=next_result,
> +                config=self.pytest_config)
>              if hook_result:
>                  for new_expl in hook_result:
>                      if new_expl:
> diff --git a/py/_plugin/hookspec.py b/py/_plugin/hookspec.py
> --- a/py/_plugin/hookspec.py
> +++ b/py/_plugin/hookspec.py
> @@ -127,7 +127,7 @@ def pytest_sessionfinish(session, exitst
>  # hooks for customising the assert methods
>  # -------------------------------------------------------------------------
>  
> -def pytest_assert_binrepr(op, left, right):
> +def pytest_assert_binrepr(op, left, right, config):
>      """Customise explanation for binary operators
>  
>      Return None or an empty list for no custom explanation, otherwise
> diff --git a/py/_plugin/pytest_assertion.py b/py/_plugin/pytest_assertion.py
> --- a/py/_plugin/pytest_assertion.py
> +++ b/py/_plugin/pytest_assertion.py
> @@ -16,7 +16,7 @@ def pytest_configure(config):
>          warn_about_missing_assertion()
>          config._oldassertion = py.builtin.builtins.AssertionError
>          py.builtin.builtins.AssertionError = py.code._AssertionError
> -        py.builtin.builtins.AssertionError._pytesthook = config.hook
> +        py.code.customise_assert(config)
>  
>  def pytest_unconfigure(config):
>      if hasattr(config, '_oldassertion'):

> _______________________________________________
> py-dev mailing list
> py-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/py-dev


-- 


From issues-noreply at bitbucket.org  Sun Oct  3 15:56:09 2010
From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org)
Date: Sun, 03 Oct 2010 13:56:09 -0000
Subject: [py-dev] New issue 125 in py-trunk: py.test exits when TypeError
	raises in __init__
Message-ID: <0293e80b053607d381ec2e6f140f597d@bitbucket.org>

--- you can reply above this line ---

New issue 125: py.test exits when TypeError raises in __init__
http://bitbucket.org/hpk42/py-trunk/issue/125/pytest-exits-when-typeerror-raises-in

Alfredo Deza / alfredodeza on Sun, 3 Oct 2010 15:56:09 +0200:

Description:
  When testing a class that raises TypeError (the test assumed the __init__ had an attribute that didn't exist) py.test exits without a traceback.

Even when running py.test in verbose mode or with --fulltrace it still doesn't report why the test is failing.

Running the test with python (e.g. no test suite) the traceback is reported correctly.

This is how it looks with py.test:
{{{
#!bash

py.test --fulltrace -v test_hg.py
================ test session starts ==============
platform darwin -- Python 2.6.1 -- pytest-1.3.4 -- /Users/alfredo/python/vpacha/bin/python
test path 1: test_hg.py

test_hg.py:43: TestHg.test_clone FAIL
test_hg.py:64: TestHg.test_commit %   
}}}
 
This is how running it with Python looks:

{{{
python test_hg.py                
EEEEEEEEEE
===========================
ERROR: Builds a mercurial repo and commits
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_hg.py", line 71, in test_commit
    conf=self.dict_conf)
TypeError: __init__() got an unexpected keyword argument 'conf'


}}}

I'm truncating the rest of tests that fail running with Python and showing the full output with py.test

Responsible:
  hpk42
-- 
This is an issue notification from bitbucket.org.
You are receiving this either because you are the
owner of the issue, or you are following the issue.


From flub at devork.be  Tue Oct  5 21:16:27 2010
From: flub at devork.be (Floris Bruynooghe)
Date: Tue, 5 Oct 2010 20:16:27 +0100
Subject: [py-dev] controlling assertions and difflib
In-Reply-To: <20101002173342.GT20695@trillke.net>
References: <AANLkTimf7aY-BczSSVvfKFcNvD5BTz6bpE67OEsWyXjM@mail.gmail.com>
	<20100929132818.GK20695@trillke.net>
	<AANLkTinY-2_d5+RqhEJ0dGXmY=PFJHwc_opQi=P-2OL5@mail.gmail.com>
	<20101002173342.GT20695@trillke.net>
Message-ID: <AANLkTik4=_UTaFo+pdQ-yBe3Bf0DYeuwqNnf+99P3i-u@mail.gmail.com>

Hello Holger

On 2 October 2010 18:33, holger krekel <holger at merlinux.eu> wrote:
> On Fri, Oct 01, 2010 at 00:23 +0100, Floris Bruynooghe wrote:
>> I've committed a cleaned-up version of this now.
>
> Great, thanks. ?It's now merged to py-trunk!

Great!

>> >> This whole issue made me wonder if it should be possible to disable
>> >> the pytest_assert_binrepr hook.
>> >
>> > We could do a general "--assertmode=(choice)" with e.g.:
>> >
>> > ? ?0: (no) don't do any assert reinterp (like --no-assert now)
>> > ? ?1: (basic) do basic reinterpreation
>> > ? ?2: (advanced) do basic reinterpretation + customizing hooks
>> >
>> > and default to 2.
>>
>> Sounds like a good idea. ?Do you think there's place for a
>> advanced-no-builtin-plugin option in here, so that the builtin hook
>> will not run any code? ?Otherwise a bug in the builtin customisation
>> hook would completely stop this from being useful.
>
> I am not so afraid but we can do it.

Ok, never mind.


> I think i meant something different. ?Please have a look into this
> changeset which i just added:
>
> ? ?http://bitbucket.org/hpk42/py-trunk/changeset/64962794201b
>
> does this make sense to you?

Yes, that makes more sense.  Very neat.


Is there still anything you'd like me to do on this?  E.g. adding the
--assertmode (and leave --no-assert as an alias for --assertmode=0).
Personally I'm pretty neutral on that in retrospect.


Thanks for merging this.
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org


From flub at devork.be  Tue Oct  5 22:22:40 2010
From: flub at devork.be (Floris Bruynooghe)
Date: Tue, 5 Oct 2010 21:22:40 +0100
Subject: [py-dev] Assert repr of attributes
Message-ID: <AANLkTikvkoQesxi1CCxCYbRFY9M25fwCAHdDLzYkCLWG@mail.gmail.com>

Hello

Another possible improvement to the assert repr of py.test (IMHO) is
how attributes are represented.  One thing I often find myself doing
is code like this:

def test_foo():
    i = SomeClass()
    print i.attribute_name
    assert i.attribute_name == "some_value"

The printing of the attribute name is simply to work around the
slightly annoying (IMHO) default representation of attributes, it
could have been a one-liner test otherwise (assuming no side-effects
in the call).  While it is really nice to see where it came from, just
knowing the value is often very useful (e.g. when it could be None or
a meaning value).

So the attached patch addresses this.  It's just a one-line change
really, and I think the output is a lot nicer.  Personally I can't see
what could be wrong with this approach so don't think moving this to a
plugin or making it otherwise optional is needed.

I hope this can be a useful addition.

Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: attrib.diff
Type: text/x-patch
Size: 2535 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20101005/b6b012ae/attachment.bin>

From issues-noreply at bitbucket.org  Wed Oct  6 10:37:27 2010
From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org)
Date: Wed, 06 Oct 2010 08:37:27 -0000
Subject: [py-dev] New issue 126 in py-trunk: Automatically disable capturing
 when pdb.set_trace() is called
Message-ID: <ab98cb353c5f54954afac06e4e3ebb45@bitbucket.org>

--- you can reply above this line ---

New issue 126: Automatically disable capturing when pdb.set_trace() is called
http://bitbucket.org/hpk42/py-trunk/issue/126/automatically-disable-capturing-when-pdbset_trace-is

Antonio Cuni / antocuni on Wed, 6 Oct 2010 10:37:27 +0200:

Description:
  Currently, if I have a pdb.set_trace() call in my code and run py.test without -s, I get an IOError instead of the pdb prompt.

Responsible:
  hpk42
-- 
This is an issue notification from bitbucket.org.
You are receiving this either because you are the
owner of the issue, or you are following the issue.


From holger at merlinux.eu  Wed Oct  6 10:55:41 2010
From: holger at merlinux.eu (holger krekel)
Date: Wed, 6 Oct 2010 10:55:41 +0200
Subject: [py-dev] Assert repr of attributes
In-Reply-To: <AANLkTikvkoQesxi1CCxCYbRFY9M25fwCAHdDLzYkCLWG@mail.gmail.com>
References: <AANLkTikvkoQesxi1CCxCYbRFY9M25fwCAHdDLzYkCLWG@mail.gmail.com>
Message-ID: <20101006085540.GY20695@trillke.net>

Hi Floris,

On Tue, Oct 05, 2010 at 21:22 +0100, Floris Bruynooghe wrote:
> Hello
> 
> Another possible improvement to the assert repr of py.test (IMHO) is
> how attributes are represented.  One thing I often find myself doing
> is code like this:
> 
> def test_foo():
>     i = SomeClass()
>     print i.attribute_name
>     assert i.attribute_name == "some_value"

Yes, this annoyed me as well a couple of times :) 
 
> The printing of the attribute name is simply to work around the
> slightly annoying (IMHO) default representation of attributes, it
> could have been a one-liner test otherwise (assuming no side-effects
> in the call).  While it is really nice to see where it came from, just
> knowing the value is often very useful (e.g. when it could be None or
> a meaning value).
> 
> So the attached patch addresses this.  It's just a one-line change
> really, and I think the output is a lot nicer.  Personally I can't see
> what could be wrong with this approach so don't think moving this to a
> plugin or making it otherwise optional is needed.

right.  There was something that was problematic IIRC but i am not sure
what it was.  And it surely is useful.  So let's apply it :) 

Can you create a bundle for this changeset 
or let me pull this change from your repo? 

> I hope this can be a useful addition.

sure!
holger


From issues-noreply at bitbucket.org  Wed Oct  6 15:25:44 2010
From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org)
Date: Wed, 06 Oct 2010 13:25:44 -0000
Subject: [py-dev] New issue 127 in py-trunk: py.test doesn't wait for
 deferred completion when using twisted.trial
Message-ID: <b26fb0f13cb60078e54766cbdd9415d6@bitbucket.org>

--- you can reply above this line ---

New issue 127: py.test doesn't wait for deferred completion when using twisted.trial
http://bitbucket.org/hpk42/py-trunk/issue/127/pytest-doesnt-wait-for-deferred-completion-when-using

Ali Afshar / aafshar on Wed, 6 Oct 2010 15:25:44 +0200:

Description:
  Twisted.trial unittests have an additional feature where if they return a deferred, the deferred will be waited on until completed. This behaviour is present in the base TestCase class twisted.trial.unittest.TestCase. These deferreds are not currently waited on in py.test.


Additionally, some twisted unittests are presented as generators decorated by  twisted.internet.defer.inlineCallbacks in which yield yields a deferreds. These generators should be iterated to completion and all generators waited upon for completion.


This is just a placeholder, as I should investigate this myself, and add more information to this ticket.

Responsible:
  hpk42
-- 
This is an issue notification from bitbucket.org.
You are receiving this either because you are the
owner of the issue, or you are following the issue.


From flub at devork.be  Wed Oct  6 19:25:00 2010
From: flub at devork.be (Floris Bruynooghe)
Date: Wed, 6 Oct 2010 18:25:00 +0100
Subject: [py-dev] Assert repr of attributes
In-Reply-To: <20101006085540.GY20695@trillke.net>
References: <AANLkTikvkoQesxi1CCxCYbRFY9M25fwCAHdDLzYkCLWG@mail.gmail.com>
	<20101006085540.GY20695@trillke.net>
Message-ID: <AANLkTinfFijffKnodX3XKd5py4dxrLNG9eAeXiDTKJ8E@mail.gmail.com>

On 6 October 2010 09:55, holger krekel <holger at merlinux.eu> wrote:

> Can you create a bundle for this changeset
> or let me pull this change from your repo?

Bundle attached (which produces slightly better output then the original patch).

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: attrib.hg
Type: application/octet-stream
Size: 1420 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20101006/e7071b96/attachment.obj>

From issues-noreply at bitbucket.org  Thu Oct  7 14:48:00 2010
From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org)
Date: Thu, 07 Oct 2010 12:48:00 -0000
Subject: [py-dev] New issue 128 in py-trunk: py.test does not show
 SyntaxErrors when using --pdb
Message-ID: <73fff333b0014f8a5b78ac10676f96e1@bitbucket.org>

--- you can reply above this line ---

New issue 128: py.test does not show SyntaxErrors when using --pdb
http://bitbucket.org/hpk42/py-trunk/issue/128/pytest-does-not-show-syntaxerrors-when-using

Antonio Cuni / antocuni on Thu, 7 Oct 2010 14:48:00 +0200:

Description:
  If the test file contains a SyntaxError, py.test --pdb silently fails without telling anything about the cause:

{{{
$ cat test_syntax_error.py 
a =
$ py.test test_syntax_error.py 
================================= test session starts ==================================
platform linux2 -- Python 2.6.5 -- pytest-1.4.0a1

/tmp/foo/test_syntax_error.py E
test path 1: test_syntax_error.py

======================================== ERRORS ========================================
____________________ ERROR collecting /tmp/foo/test_syntax_error.py ____________________
/home/antocuni/pypy/misc/py-trunk/py/_plugin/pytest_python.py:216: in _importtestmodule
>           mod = self.fspath.pyimport(ensuresyspath=True)
/home/antocuni/pypy/misc/py-trunk/py/_path/local.py:530: in pyimport
>           mod = __import__(modname, None, None, ['__doc__'])
E             File "/tmp/foo/test_syntax_error.py", line 1
E               a =
E                 ^
E           SyntaxError: invalid syntax
=============================== 1 error in 0.02 seconds ================================
$ py.test test_syntax_error.py --pdb
================================= test session starts ==================================
platform linux2 -- Python 2.6.5 -- pytest-1.4.0a1

/tmp/foo/test_syntax_error.py E
test path 1: test_syntax_error.py

=============================== 1 error in 0.04 seconds ================================
$ 
}}}




Responsible:
  hpk42
-- 
This is an issue notification from bitbucket.org.
You are receiving this either because you are the
owner of the issue, or you are following the issue.


From holger at merlinux.eu  Sun Oct 10 15:03:44 2010
From: holger at merlinux.eu (holger krekel)
Date: Sun, 10 Oct 2010 15:03:44 +0200
Subject: [py-dev] the split is happening: py -> pytest,pylib,pycmd
Message-ID: <20101010130343.GC11924@trillke.net>

Hi all,

if you happen to pull/update from the bitbucket "py-trunk" repo
you'll find it contains almost no files anymore but rather has a setup.py
that specifies dependencies on the following split out distributions:

    * pytest: https://bitbucket.org/hpk42/pytest

    * pylib: https://bitbucket.org/hpk42/pylib

    * pycmd: https://bitbucket.org/hpk42/pycmd

Nothing is released to PYPI yet and there are certainly some rough edges.

However, you should be able to grab the above repos and issue
"python setup.py develop" in each (first on pylib, then pytest).

I am currently working a lot on documentation and further 
refinements to packaging.   Feedback/comments welcome.

best,
holger


From anto.cuni at gmail.com  Sun Oct 10 15:12:17 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sun, 10 Oct 2010 15:12:17 +0200
Subject: [py-dev] the split is happening: py -> pytest,pylib,pycmd
In-Reply-To: <20101010130343.GC11924@trillke.net>
References: <20101010130343.GC11924@trillke.net>
Message-ID: <4CB1BBB1.3090301@gmail.com>

Hi!

On 10/10/10 15:03, holger krekel wrote:
> Hi all,
>
> if you happen to pull/update from the bitbucket "py-trunk" repo
> you'll find it contains almost no files anymore but rather has a setup.py
> that specifies dependencies on the following split out distributions:
>
>      * pytest: https://bitbucket.org/hpk42/pytest
>
>      * pylib: https://bitbucket.org/hpk42/pylib
>
>      * pycmd: https://bitbucket.org/hpk42/pycmd

I've quickly looked at the repos, but there is something that I don't get: I 
see that there is still pycmd/pycmd/pytest.py, however pytest is not listed in 
the dependencies.

Does it simply mean that you forgot to remove pytest.py from pycmd, or that 
pycmd will be required to get the py.test executable?

ciao,
Anto


From holger at merlinux.eu  Sun Oct 10 15:17:01 2010
From: holger at merlinux.eu (holger krekel)
Date: Sun, 10 Oct 2010 15:17:01 +0200
Subject: [py-dev] the split is happening: py -> pytest,pylib,pycmd
In-Reply-To: <4CB1BBB1.3090301@gmail.com>
References: <20101010130343.GC11924@trillke.net> <4CB1BBB1.3090301@gmail.com>
Message-ID: <20101010131701.GD11924@trillke.net>

On Sun, Oct 10, 2010 at 15:12 +0200, Antonio Cuni wrote:
> On 10/10/10 15:03, holger krekel wrote:
>> Hi all,
>>
>> if you happen to pull/update from the bitbucket "py-trunk" repo
>> you'll find it contains almost no files anymore but rather has a setup.py
>> that specifies dependencies on the following split out distributions:
>>
>>      * pytest: https://bitbucket.org/hpk42/pytest
>>
>>      * pylib: https://bitbucket.org/hpk42/pylib
>>
>>      * pycmd: https://bitbucket.org/hpk42/pycmd
>
> I've quickly looked at the repos, but there is something that I don't 
> get: I see that there is still pycmd/pycmd/pytest.py, however pytest is 
> not listed in the dependencies.

thanks, removed.

> Does it simply mean that you forgot to remove pytest.py from pycmd, or 
> that pycmd will be required to get the py.test executable?

i forgot it. there may be other places like this. Thanks for looking.
best,
holger


From issues-noreply at bitbucket.org  Tue Oct 12 17:32:46 2010
From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org)
Date: Tue, 12 Oct 2010 15:32:46 -0000
Subject: [py-dev] New issue 129 in py-trunk: Test file names of the form
 foo_bar___init__.py error in 1.3.2, 1.3.3 and 1.3.4
Message-ID: <c02f3ad338bb4aac48aa4253f934ea38@bitbucket.org>

--- you can reply above this line ---

New issue 129: Test file names of the form foo_bar___init__.py error in 1.3.2, 1.3.3 and 1.3.4
http://bitbucket.org/hpk42/py-trunk/issue/129/test-file-names-of-the-form-foo_bar___init__py-error-in-132-133-and

trevorbaca on Tue, 12 Oct 2010 17:32:46 +0200:

Description:
  Hi,

Starting in 1.3.2 test file names *ending with* the string '__init__.py' cause py.test to fail.

This would seem to relate to #119 (fixed in 1.3.4). But this version of the problem persists in the 1.3.4 public release.

In the example below, the file foo_bar___init__.py contains just a single print statement:


Desktop$ py.test foo_bar__init__.py 
======================================= test session starts ========================================
platform darwin -- Python 2.6.1 -- pytest-1.3.4
test path 1: foo_bar__init__.py

foo_bar__init__.py E

============================================== ERRORS ==============================================
__________________ ERROR collecting /Users/trevorbaca/Desktop/foo_bar__init__.py ___________________
/Library/Python/2.6/site-packages/py-1.3.4-py2.6.egg/py/_plugin/pytest_runner.py:34: in pytest_make_collect_report
>           result = collector._memocollect()
/Library/Python/2.6/site-packages/py-1.3.4-py2.6.egg/py/_test/collect.py:230: in _memocollect
>       return self._memoizedcall('_collected', self.collect)
/Library/Python/2.6/site-packages/py-1.3.4-py2.6.egg/py/_test/collect.py:104: in _memoizedcall
>           res = function()
/Library/Python/2.6/site-packages/py-1.3.4-py2.6.egg/py/_test/pycollect.py:82: in collect
>       dicts = [getattr(self.obj, '__dict__', {})]
/Library/Python/2.6/site-packages/py-1.3.4-py2.6.egg/py/_test/pycollect.py:17: in fget
>           self._obj = obj = self._getobj()
/Library/Python/2.6/site-packages/py-1.3.4-py2.6.egg/py/_test/pycollect.py:140: in _getobj
>       return self._memoizedcall('_obj', self._importtestmodule)
/Library/Python/2.6/site-packages/py-1.3.4-py2.6.egg/py/_test/collect.py:104: in _memoizedcall
>           res = function()
/Library/Python/2.6/site-packages/py-1.3.4-py2.6.egg/py/_test/pycollect.py:145: in _importtestmodule
>           mod = self.fspath.pyimport(ensuresyspath=True)
/Library/Python/2.6/site-packages/py-1.3.4-py2.6.egg/py/_path/local.py:539: in pyimport
>           if not self.samefile(modfile):
/Library/Python/2.6/site-packages/py-1.3.4-py2.6.egg/py/_path/local.py:73: in samefile
>       return py.error.checked_call(os.path.samefile, str(self), str(other))
/Library/Python/2.6/site-packages/py-1.3.4-py2.6.egg/py/_error.py:80: in checked_call
>           raise cls("%s%r" % (func.__name__, args))
E           ENOENT: [No such file or directory]: samefile('/Users/trevorbaca/Desktop/foo_bar__init__.py', '/Users/trevorbaca/Desktop/foo_ba')
===================================== 1 error in 0.09 seconds ======================================


This matters for us at www.projectabjad.org because we author tests for the initializers of all of our classes in files named as FooBarClass___init__.py.

Trevor.

Responsible:
  hpk42
-- 
This is an issue notification from bitbucket.org.
You are receiving this either because you are the
owner of the issue, or you are following the issue.


From fijall at gmail.com  Tue Oct 12 18:31:09 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 12 Oct 2010 18:31:09 +0200
Subject: [py-dev] do you mind the issue-postings here?
In-Reply-To: <20100903153034.GE32478@trillke.net>
References: <20100903153034.GE32478@trillke.net>
Message-ID: <AANLkTinGNMMLZPPvg_8E6oy6O81ysGTDnvj3-Ben+zr4@mail.gmail.com>

personally, I think a separate mailing list would be better (I've
started to ignore py-dev and I think issues played major part), but I
can only speak about myself.

On Fri, Sep 3, 2010 at 5:30 PM, holger krekel <holger at merlinux.eu> wrote:
> Hi all,
>
> am wondering: do you mind seeing new bitbucket issue
> creations here on the mailing list? ?If so i'll create
> a new mailing list pypy-issue or so and have things
> rather sent there. Just let me know.
>
> Personally i find it a bit unfortuante that the many
> closed/resolved issues are not notified to this list as well :)
>
> cheers,
> holger
> _______________________________________________
> py-dev mailing list
> py-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/py-dev
>


From holger at merlinux.eu  Fri Oct 15 01:13:15 2010
From: holger at merlinux.eu (holger krekel)
Date: Fri, 15 Oct 2010 01:13:15 +0200
Subject: [py-dev] installing new py.test / new pypi.testrun.org
Message-ID: <20101014231315.GR11924@trillke.net>

Hi, 

i created a simple PyPI server instance as a playground and to make 
it easier to test interdependent packagages i am working on.  See

    http://pypi.testrun.org

You can use it to install the latest in-development version of pytest,
tox, pylib, pycmd, pytest-xdist etc. (*) with e. g.:

    pip install -i http://pypi.testrun.org pytest 

or tox::

    pip install -i http://pypi.testrun.org tox

These are _development_ versions so they might break more easily.
But they also fix a number of bugs and provide some new features :)

cheers,
holger



From holger at merlinux.eu  Fri Oct 15 20:14:26 2010
From: holger at merlinux.eu (holger krekel)
Date: Fri, 15 Oct 2010 20:14:26 +0200
Subject: [py-dev] pytest / pylib / tox release splitting plan
Message-ID: <20101015181425.GZ11924@trillke.net>

Hi all,

here are some py.test/pylib/tox split release related stories i have been
thinking about.  Idea is to allow all kinds of installation / upgrade situations
to succeed.  My goal is to have "tox" only pull in the new small and mean
"pylib" and not the whole pytest package. Another is to have a small mostly
self-contained "pytest" package. And a third to seperate out the helper
tools into a "pycmd" package.  

Anyone who is interested in release management might want to read this,
and maybe help me see the light or be inspired ... all others can just 
hope i get this right and don't mess up :) 

consider this situation:

    pip install py     # you currently get sumo py==1.3.4, for using py.test
    pip install pylib  # will be pylib==2.0.0

this would lead to "import py" coming from py==1.3.4 because the distribution
'py' "owns py" and came first.  You'd also see funny tracebacks when invoking
py.test. BAD.

My planned fix is to provide a py==1.4.0 which has _no_ packages
and have pylib depend on the exact "py==1.4.0" version.  Then

    pip install pylib  

will lead to "import py" coming from pylib-2.0.0 because the py==1.4.0 
is an upgrade to 1.3.4, thus removes its "py", package, no interference.  Good.

Now consider a tox user, typing::

    pip install tox   # let's assume this is 0.8

tox has a requirement of "py" and if this drags py==1.4.0 - BUMM!
the tox installs breaks, because when it "import py" nothing is there.

My planned fix is to provide *another* release "py==1.4.1" which will depend on
pylib-2.0 and pytest-2.0 (and pycmd-1.0) so that people blindly installing  tox
like above will be happy and people doing 

    pip install -U py

and expecting to get py.test (because they are reading an old README
or blog post) will also be happy and use the newest version of pytest/pylib.

Finally, somebody starting a fresh installation with

    pip install pytest 

will just drag in pylib-2.0, which drags in py==1.4.0 which requires
pylib-2.0 ... SIMPLE, good.

You might wonder why i care so much and don't simply tell people
"make sure you uninstall py first" or something like this.

Reason: many people are using the above packages in Continous
Integration situations and going to all kinds of masters, slaves
and Job configurations to ensure removal of packages and such
is not fun.  And not everybody reads release messages anyway ...
or understands what is installed in their environments - judging
from myself :) 

best,
holger


From anto.cuni at gmail.com  Sat Oct 16 19:05:29 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sat, 16 Oct 2010 19:05:29 +0200
Subject: [py-dev] Detect whether py.test is running
Message-ID: <4CB9DB59.50501@gmail.com>

Hi,

I need a way to distinguish whether the current module is being imported 
because we are running the tests, of whether we are running the main program.

I know that the correct way would be to inject the information from the top, 
but I'm using a framework which relies on some global state to store the 
various options, and I have to deal with it.

Anyway, is there any official/recommended way to do it?
What I'm doing right now is this (py 1.3.1):

if py.test.config.__dict__ == {}:
     # py.test is NOT running
     ...
else:
     # py.test is running

ciao,
Anto


From anto.cuni at gmail.com  Sat Oct 16 19:09:18 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sat, 16 Oct 2010 19:09:18 +0200
Subject: [py-dev] Detect whether py.test is running
In-Reply-To: <4CB9DB59.50501@gmail.com>
References: <4CB9DB59.50501@gmail.com>
Message-ID: <4CB9DC3E.9040901@gmail.com>

On 16/10/10 19:05, Antonio Cuni wrote:

> if py.test.config.__dict__ == {}:

sorry, it's py.test.config.option.__dict__

(uhm, what about providing a way to test the code inside the emails before 
sending them? :-))


From holger at merlinux.eu  Sat Oct 16 22:56:43 2010
From: holger at merlinux.eu (holger krekel)
Date: Sat, 16 Oct 2010 22:56:43 +0200
Subject: [py-dev] Detect whether py.test is running
In-Reply-To: <4CB9DB59.50501@gmail.com>
References: <4CB9DB59.50501@gmail.com>
Message-ID: <20101016205643.GC11924@trillke.net>

Hi Anto, all,

First of all, please keep asking "usage" questions or ask for examples
for practical / real life problems you have.  They help me filling
the example sections of the new py.test docs with useful material :)

On Sat, Oct 16, 2010 at 19:05 +0200, Antonio Cuni wrote:
> I need a way to distinguish whether the current module is being imported 
> because we are running the tests, of whether we are running the main program.
> 
> I know that the correct way would be to inject the information from the top, 
> but I'm using a framework which relies on some global state to store the 
> various options, and I have to deal with it.

Right, which is why i wouldn't like this to be totally convenient :)

> Anyway, is there any official/recommended way to do it?
> What I'm doing right now is this (py 1.3.1):
> 
> if py.test.config.__dict__ == {}:
>      # py.test is NOT running
>      ...
> else:
>      # py.test is running

uh, hacky :)  also will break with pytest-2.0 as the py.test.config
will not be there outside of test runs.

I drafted an example which should also work with py-1.3.1 as part of
the examples section of the upcoming pytest docs:

    http://codespeak.net/~hpk/pytest/example/misc.html

Works for you?

cheers,
holger


From red_socks at gmx.de  Sun Oct 17 10:59:45 2010
From: red_socks at gmx.de (Frank Hempel)
Date: Sun, 17 Oct 2010 10:59:45 +0200
Subject: [py-dev] Detect whether py.test is running
In-Reply-To: <20101016205643.GC11924@trillke.net>
References: <4CB9DB59.50501@gmail.com> <20101016205643.GC11924@trillke.net>
Message-ID: <4CBABB01.3050304@gmx.de>

Am 16.10.2010 22:56, schrieb holger krekel:
...
> I drafted an example which should also work with py-1.3.1 as part of
> the examples section of the upcoming pytest docs:
> 
>     http://codespeak.net/~hpk/pytest/example/misc.html

I guess it's a typo:

"Usually it is a good idea to make application code behave differently
if called from a test."

It's usually a bad idea, isn't it?

Regards, Frank


From holger at merlinux.eu  Sun Oct 17 11:11:16 2010
From: holger at merlinux.eu (holger krekel)
Date: Sun, 17 Oct 2010 11:11:16 +0200
Subject: [py-dev] Detect whether py.test is running
In-Reply-To: <4CBABB01.3050304@gmx.de>
References: <4CB9DB59.50501@gmail.com> <20101016205643.GC11924@trillke.net>
	<4CBABB01.3050304@gmx.de>
Message-ID: <20101017091116.GD11924@trillke.net>

On Sun, Oct 17, 2010 at 10:59 +0200, Frank Hempel wrote:
> Am 16.10.2010 22:56, schrieb holger krekel:
> ...
> > I drafted an example which should also work with py-1.3.1 as part of
> > the examples section of the upcoming pytest docs:
> > 
> >     http://codespeak.net/~hpk/pytest/example/misc.html
> 
> I guess it's a typo:
> 
> "Usually it is a good idea to make application code behave differently
> if called from a test."
> 
> It's usually a bad idea, isn't it?

right. fixed. thanks!
holger


From anto.cuni at gmail.com  Sun Oct 17 22:24:46 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sun, 17 Oct 2010 22:24:46 +0200
Subject: [py-dev] Detect whether py.test is running
In-Reply-To: <20101016205643.GC11924@trillke.net>
References: <4CB9DB59.50501@gmail.com> <20101016205643.GC11924@trillke.net>
Message-ID: <4CBB5B8E.3000909@gmail.com>

Hi Holger, hi all

On 16/10/10 22:56, holger krekel wrote:
> Hi Anto, all,
>
> First of all, please keep asking "usage" questions or ask for examples
> for practical / real life problems you have.  They help me filling
> the example sections of the new py.test docs with useful material :)

sorry, I don't get it. Do you mean that I should describe my problem in more 
details? Ok, here is it.

I am developing a program using scrapy (to get data from web pages). Following 
the (bad?) habit of lot of frameworks, scrapy requires you to define some 
settings as global variables in a settings.py module.  In particular, the 
LOG_FILE variable determine where the log is written.

In my program, I would like the log file to be placed in a directory named 
after the date, e.g. '2010-10-17/myprogram.log'.  So, my settings.py contains 
something like:

LOGDIR = BASEDIR.join(str(datetime.date.today()))
LOGDIR.ensure(dir=True)
LOG_FILE = LOGDIR.join('myprogram.log')

The issue is simply that I don't want to create the directory when I run the 
test, as the log file is not needed (and even if it were, I'd want it to be in 
a separate tmpdir, not in the main directory).

All the other solutions I could think of require to change the code in scrapy 
itself, which I'd prefer not to do, of course.


> Right, which is why i wouldn't like this to be totally convenient :)

+1 :-)


> uh, hacky :)  also will break with pytest-2.0 as the py.test.config
> will not be there outside of test runs.
>
> I drafted an example which should also work with py-1.3.1 as part of
> the examples section of the upcoming pytest docs:
>
>      http://codespeak.net/~hpk/pytest/example/misc.html
>
> Works for you?

I've not tried it yet, but should work and it's much better than my current 
approach :-). Thanks!

ciao,
Anto


From holger at merlinux.eu  Wed Oct 20 09:57:54 2010
From: holger at merlinux.eu (holger krekel)
Date: Wed, 20 Oct 2010 09:57:54 +0200
Subject: [py-dev] Detect whether py.test is running
In-Reply-To: <4CBB5B8E.3000909@gmail.com>
References: <4CB9DB59.50501@gmail.com> <20101016205643.GC11924@trillke.net>
	<4CBB5B8E.3000909@gmail.com>
Message-ID: <20101020075754.GE11924@trillke.net>

On Sun, Oct 17, 2010 at 22:24 +0200, Antonio Cuni wrote:
> Hi Holger, hi all
>
> On 16/10/10 22:56, holger krekel wrote:
>> Hi Anto, all,
>>
>> First of all, please keep asking "usage" questions or ask for examples
>> for practical / real life problems you have.  They help me filling
>> the example sections of the new py.test docs with useful material :)
>
> sorry, I don't get it. Do you mean that I should describe my problem in 
> more details? Ok, here is it.

Sorry for being unclear.  I just meant that i welcome your mail
and encourage everyone to ask similar questions.

> I am developing a program using scrapy (to get data from web pages). 
> Following the (bad?) habit of lot of frameworks, scrapy requires you to 
> define some settings as global variables in a settings.py module.  In 
> particular, the LOG_FILE variable determine where the log is written.
>
> In my program, I would like the log file to be placed in a directory 
> named after the date, e.g. '2010-10-17/myprogram.log'.  So, my 
> settings.py contains something like:
>
> LOGDIR = BASEDIR.join(str(datetime.date.today()))
> LOGDIR.ensure(dir=True)
> LOG_FILE = LOGDIR.join('myprogram.log')
>
> The issue is simply that I don't want to create the directory when I run 
> the test, as the log file is not needed (and even if it were, I'd want it 
> to be in a separate tmpdir, not in the main directory).
>
> All the other solutions I could think of require to change the code in 
> scrapy itself, which I'd prefer not to do, of course.
>
>
>> Right, which is why i wouldn't like this to be totally convenient :)
>
> +1 :-)
>
>
>> uh, hacky :)  also will break with pytest-2.0 as the py.test.config
>> will not be there outside of test runs.
>>
>> I drafted an example which should also work with py-1.3.1 as part of
>> the examples section of the upcoming pytest docs:
>>
>>      http://codespeak.net/~hpk/pytest/example/misc.html
>>
>> Works for you?
>
> I've not tried it yet, but should work and it's much better than my 
> current approach :-). Thanks!

good :)
cheers,
holger


From issues-noreply at bitbucket.org  Wed Oct 20 11:30:44 2010
From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org)
Date: Wed, 20 Oct 2010 09:30:44 -0000
Subject: [py-dev] New issue 130 in py-trunk: py.process.cmdexec fails if the
 out/err contains non-ascii characters
Message-ID: <58ef562c5a9bae1fe483285f47a4ed6e@bitbucket.org>

--- you can reply above this line ---

New issue 130: py.process.cmdexec fails if the out/err contains non-ascii characters
http://bitbucket.org/hpk42/py-trunk/issue/130/pyprocesscmdexec-fails-if-the-out-err-contains-non-ascii

Antonio Cuni / antocuni on Wed, 20 Oct 2010 11:30:44 +0200:

Description:
  Consider the following file trouble.py, which just output a non-ascii character to stdout:
{{{
# -*- encoding: utf-8 -*-
trouble = u'?'
print trouble.encode('utf-8')
}}}

py.process.cmdexec fails if we try to run it:
{{{
>>> import py
>>> py.process.cmdexec('python trouble.py')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/antocuni/pypy/misc/py-trunk/py/_process/cmdexec.py", line 32, in cmdexec
    raise ExecutionFailed(status, status, cmd, out, err)
py.process.cmdexec.Error: ExecutionFailed: 1  python trouble.py
Traceback (most recent call last):
  File "trouble.py", line 3, in <module>
    print trouble
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe0' in position 0: ordinal not in range(128)
}}}

This happens because cmdexec try to decode stderr and stdout using sys.stdout.encoding or sys.getdefaultencoding(), which not necessarly match the output encoding of the program.

Real life use-case: a pypy test invokes gcc, which uses utf-8 characters for quotes. The test fails because of the exception inside cmdexec, even if the test itself completely ignores the stderr.

Responsible:
  hpk42
-- 
This is an issue notification from bitbucket.org.
You are receiving this either because you are the
owner of the issue, or you are following the issue.


From eli at water.ca.gov  Thu Oct 28 02:00:58 2010
From: eli at water.ca.gov (Ateljevich, Eli)
Date: Wed, 27 Oct 2010 17:00:58 -0700
Subject: [py-dev] py.test to control other testing framework
Message-ID: <67CC54752B3F8D43ACD7E3AD4551626306C423E88F@mrsbmapp20306.ad.water.ca.gov>

On another list, I asked Holger about how to use py.test as a wrapper to control non-python tests. He referred me to this example:
self-contained yaml example<http://codespeak.net/~hpk/pytest/example/nonpython.html>

This got me going on the issue of collecting and running the tests.

I have a follow-up question about aggregating results from these "foreign tests". I am using cxxunit, but the specifics are not important to my question. Each non-python test involves multiple asserts, failures and possibly errors. These test results could be reported in any of the usual formats (logs, stdout, JUnit xml format, custom).

My question is this: is there a good way to track assert pass/fail counts correctly in py.test?

One crude idea is to have cxxtest print out its assert attempts, passes and failures to a log, parse the log and then deliberately pass and fail a correct number of "shadow assertions" in python using the same log messages. Is there a more direct way?

Thanks,
Eli

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20101027/9cff11d9/attachment.html>

From prologic at shortcircuit.net.au  Thu Oct 28 02:17:21 2010
From: prologic at shortcircuit.net.au (James Mills)
Date: Thu, 28 Oct 2010 10:17:21 +1000
Subject: [py-dev] py.test with setuptools
Message-ID: <AANLkTi=oRMibmRCtuUojgBxTCbSARK+w44eMC-iaW7XB@mail.gmail.com>

Hey all,

Just curious... How do you setup your setup.py so that "python
setup.py tests" works ?

cheers
James

-- 
-- James Mills
--
-- "Problems are solved by method"


From holger at merlinux.eu  Thu Oct 28 08:52:35 2010
From: holger at merlinux.eu (holger krekel)
Date: Thu, 28 Oct 2010 08:52:35 +0200
Subject: [py-dev] py.test to control other testing framework
In-Reply-To: <67CC54752B3F8D43ACD7E3AD4551626306C423E88F@mrsbmapp20306.ad.water.ca.gov>
References: <67CC54752B3F8D43ACD7E3AD4551626306C423E88F@mrsbmapp20306.ad.water.ca.gov>
Message-ID: <20101028065234.GS11924@trillke.net>

Hi Eli,

On Wed, Oct 27, 2010 at 17:00 -0700, Ateljevich, Eli wrote:
> On another list, I asked Holger about how to use py.test as a wrapper to control non-python tests. He referred me to this example:
> self-contained yaml example<http://codespeak.net/~hpk/pytest/example/nonpython.html>
> 
> This got me going on the issue of collecting and running the tests.
> 
> I have a follow-up question about aggregating results from these "foreign tests". I am using cxxunit, but the specifics are not important to my question. Each non-python test involves multiple asserts, failures and possibly errors. These test results could be reported in any of the usual formats (logs, stdout, JUnit xml format, custom).

(sidenote: I guess you are aware of the --junitxml option.)

> My question is this: is there a good way to track assert pass/fail counts correctly in py.test?
> 
> One crude idea is to have cxxtest print out its assert attempts, passes and failures to a log, parse the log and then deliberately pass and fail a correct number of "shadow assertions" in python using the same log messages. Is there a more direct way?

Parsing a log sounds right.  However, there currently is no notion of "counting
asserts" in py.test (or other popular python testing frameworks i am aware of).

Do you happen to have a way to find out the number of test functions/cases
ahead of running a test file?  And to instruct cxxtest to run a particular
function?

If so you could map cxx test functions to py.test Items and get
more fine-granular "." printing and error representation.

Otherwise i guess you can only represent the whole cxxtest file as
one test item.  Also because of other scenarios i am wondering how/if
to help this situation, btw.  

cheers,
holger


From holger at merlinux.eu  Thu Oct 28 09:34:37 2010
From: holger at merlinux.eu (holger krekel)
Date: Thu, 28 Oct 2010 09:34:37 +0200
Subject: [py-dev] py.test with setuptools
In-Reply-To: <AANLkTi=oRMibmRCtuUojgBxTCbSARK+w44eMC-iaW7XB@mail.gmail.com>
References: <AANLkTi=oRMibmRCtuUojgBxTCbSARK+w44eMC-iaW7XB@mail.gmail.com>
Message-ID: <20101028073436.GT11924@trillke.net>

Hey James,

On Thu, Oct 28, 2010 at 10:17 +1000, James Mills wrote:
> Hey all,
> 
> Just curious... How do you setup your setup.py so that "python
> setup.py tests" works ?

I've written up an example of what i think is currently the best way 
to go about it for all distutils (or setuptools) based projects:

    http://codespeak.net/~hpk/pytest/goodpractises.html#integrating-with-distutils-python-setup-py-test

Another option probably is to hack a bit deeper into setuptools and/or 
to wrap a py.test run into unittest "suites".  The latter is maybe worth
to try out but the above documented method is very solid and avoids hassles
i think.

best,
holger
 
> cheers
> James
> 
> -- 
> -- James Mills
> --
> -- "Problems are solved by method"
> _______________________________________________
> py-dev mailing list
> py-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/py-dev
> 

-- 


From holger at merlinux.eu  Fri Oct 29 00:00:54 2010
From: holger at merlinux.eu (holger krekel)
Date: Fri, 29 Oct 2010 00:00:54 +0200
Subject: [py-dev] py.test to control other testing framework
In-Reply-To: <67CC54752B3F8D43ACD7E3AD4551626306C42E302C@mrsbmapp20306.ad.water.ca.gov>
References: <20101028065234.GS11924@trillke.net>
	<67CC54752B3F8D43ACD7E3AD4551626306C42E302C@mrsbmapp20306.ad.water.ca.gov>
Message-ID: <20101028220054.GV11924@trillke.net>

Hi Eli,

On Thu, Oct 28, 2010 at 14:11 -0700, Ateljevich, Eli wrote:
> Thanks Holger. I said wanted to count assertions but I was just being careless -- counting test failures and passes the usual way is fine. 
> 
> You seem to understand my timing issue by asking "do I know how many tests there are"?  I'll ask the cxxtest folks, but I haven't seen it in their usage examples and there is no obvious --count-tests type of flag. Python would see each cxxtest suite as a monolith.
> 
> One alternative is to lie to the py.test reporting mechanism. If I understand that code snippet you pointed out, I get a shot at this in repr_failure(self, excinfo). Is excinfo an opportunity to manipulate test attributes enough to increase the total, success and fail counts:
> 1. Collector discovers cxxtests
> 2. Runner runs them
> 3. Runner parses the success/fail results and stashes them
> 4. Runner lies in repr_failure(self,excinfo), upping the counts
> Even if this is possible, it seems like at most I can do this on failure -- the number of tests and successes would be hard to count correctly. 

You can't really "lie in repr_failure" regarding test counts.   We could maybe
think about allowing multiple reports per test item. Haven't tried this
and am a bit skeptical about it producing internal problems.  It might be
surprising for other plugins like the junitxml plugin or the terminal
plugin itself which assume they get one report per collected test item IIRC.
However, it's worth an experiment i guess especially since Ronny (another
py.test hacker and heavy user) recently arrived at a similar need, it seems.
 
> The other alternative is to run and count the tests at collect time:
> 1. Collector runs the cxxtests
> 2. Collector parses the test results and emits shadow tests that trivially fail, succeed or have an error.
> 3. Test runner runs the shadow tests.
> I am wary of running cxxtests during "collect", but I can't really say why.
>
> What do you think?

Right, it doesn't look completely fitting.  Running tests at collect time 
sounds wrong.  

If i were you I'd probably first try to collect one test item per test file and
parse the log/result file, raise a failure if anything failed and show details
if any test/assertion fails.   I'd probably first perfect this (including
properly and robustly parsing the log) and see how the resulting testing process
feels in practise.  The involved coding effort is worthwhile in any case.

If allowing multiple results still feels important maybe I can try / experiment
with the above idea (somehow allowing multiple results per one test item) 
or you could try this: a test run could create a cache file which will
be found by subsequent collections which will then create multiple items 
and not just one. The items can - at runtest() time - easily share the 
same result log from a cxx-run.  

cheers,
holger

P.S.: let's keep the discussion on py-dev as this may be interesting
for future reference or other people.

> Thanks so much for following up with this!!!
> 
> Eli
> 
> 
> 
> ________________________________________
> From: holger krekel [holger at merlinux.eu]
> Sent: Wednesday, October 27, 2010 11:52 PM
> To: Ateljevich, Eli
> Cc: py-dev at codespeak.net
> Subject: Re: [py-dev] py.test to control other testing framework
> 
> Hi Eli,
> 
> On Wed, Oct 27, 2010 at 17:00 -0700, Ateljevich, Eli wrote:
> > On another list, I asked Holger about how to use py.test as a wrapper to control non-python tests. He referred me to this example:
> > self-contained yaml example<http://codespeak.net/~hpk/pytest/example/nonpython.html>
> >
> > This got me going on the issue of collecting and running the tests.
> >
> > I have a follow-up question about aggregating results from these "foreign tests". I am using cxxunit, but the specifics are not important to my question. Each non-python test involves multiple asserts, failures and possibly errors. These test results could be reported in any of the usual formats (logs, stdout, JUnit xml format, custom).
> 
> (sidenote: I guess you are aware of the --junitxml option.)
> 
> > My question is this: is there a good way to track assert pass/fail counts correctly in py.test?
> >
> > One crude idea is to have cxxtest print out its assert attempts, passes and failures to a log, parse the log and then deliberately pass and fail a correct number of "shadow assertions" in python using the same log messages. Is there a more direct way?
> 
> Parsing a log sounds right.  However, there currently is no notion of "counting
> asserts" in py.test (or other popular python testing frameworks i am aware of).
> 
> Do you happen to have a way to find out the number of test functions/cases
> ahead of running a test file?  And to instruct cxxtest to run a particular
> function?
> 
> If so you could map cxx test functions to py.test Items and get
> more fine-granular "." printing and error representation.
> 
> Otherwise i guess you can only represent the whole cxxtest file as
> one test item.  Also because of other scenarios i am wondering how/if
> to help this situation, btw.
> 
> cheers,
> holger

-- 


From prologic at shortcircuit.net.au  Fri Oct 29 01:07:31 2010
From: prologic at shortcircuit.net.au (James Mills)
Date: Fri, 29 Oct 2010 09:07:31 +1000
Subject: [py-dev] py.test with setuptools
In-Reply-To: <20101028073436.GT11924@trillke.net>
References: <AANLkTi=oRMibmRCtuUojgBxTCbSARK+w44eMC-iaW7XB@mail.gmail.com>
	<20101028073436.GT11924@trillke.net>
Message-ID: <AANLkTinFrvn5t12P=uqFUhfLiorCSz=tk3h_Y9-4x3Z6@mail.gmail.com>

On Thu, Oct 28, 2010 at 5:34 PM, holger krekel <holger at merlinux.eu> wrote:
> I've written up an example of what i think is currently the best way
> to go about it for all distutils (or setuptools) based projects:
>
> ? ?http://codespeak.net/~hpk/pytest/goodpractises.html#integrating-with-distutils-python-setup-py-test
>
> Another option probably is to hack a bit deeper into setuptools and/or
> to wrap a py.test run into unittest "suites". ?The latter is maybe worth
> to try out but the above documented method is very solid and avoids hassles
> i think.

Thanks Holger, I'll look at both methods.

cheers
James

-- 
-- James Mills
--
-- "Problems are solved by method"