From florian.proff.schulze at gmx.net  Mon May  2 12:44:47 2005
From: florian.proff.schulze at gmx.net (Florian Schulze)
Date: Mon, 02 May 2005 12:44:47 +0200
Subject: [py-dev] svnwc depends on locale
Message-ID: <opsp49gxupttxc4i@mail.gmx.net>

Hi!

I just found out that py.path.svnwc depends on the locale. I'm using 
Windows XP with svn 1.1.3 (the installer one) with german locale. As the 
functions in svnwc parse the output of the commandline utilities, they 
don't work at all, because they depend on the english output. Any idea how 
to fix this without the need to delete the message catalogs?

Regards,
Florian Schulze



From hpk at trillke.net  Mon May  2 13:05:59 2005
From: hpk at trillke.net (holger krekel)
Date: Mon, 2 May 2005 13:05:59 +0200
Subject: [py-dev] svnwc depends on locale
In-Reply-To: <opsp49gxupttxc4i@mail.gmx.net>
References: <opsp49gxupttxc4i@mail.gmx.net>
Message-ID: <20050502110559.GO22996@solar.trillke.net>

Hi Florian! 

On Mon, May 02, 2005 at 12:44 +0200, Florian Schulze wrote:
> I just found out that py.path.svnwc depends on the locale. I'm using 
> Windows XP with svn 1.1.3 (the installer one) with german locale. As the 
> functions in svnwc parse the output of the commandline utilities, they 
> don't work at all, because they depend on the english output. Any idea how 
> to fix this without the need to delete the message catalogs?

There is a commented hack at the very end of py/path/svnwc/svncommon.py 
which is invoked before issuing svn commands.  Apart from that it is
currently not in use (i didn't test it properly yet although it 
work at some point) i don't know of a solution for win32. 
Is there a way to modify the locale when running a command
on win32? 

cheers, 

    holger


From micktwomey at gmail.com  Mon May  2 15:07:13 2005
From: micktwomey at gmail.com (Michael Twomey)
Date: Mon, 2 May 2005 14:07:13 +0100
Subject: [py-dev] py.test and darcs
Message-ID: <50a522ca050502060728371abe@mail.gmail.com>

Hi,

I've been setting up a new pet project using py.test as the testing
framework, and darcs as the scm. I've hit a problem after my first
"darcs record" (darcs checkin). Darcs keeps a copy of the checked out
originals under a _darcs directory in the root of the source tree,
which causes py.test some confusion during the collect phase, since it
has duplicate python modules to contend with.

Ideally I'd like py.test to ignore _darcs (and I think {arch} for GNU
arch too) out of the box, but is there a way to twist conftest.py's
arm to make it ignore these directories?

A simple workaround is to specify my package directly. Of course, that
only works with my layout, I keep all the python files together in one
place.

(Oh yeah, py.test is very nice to use, keep up the good work, can't
wait for the doctest stuff :) )

cheers,
  Michael


My layout (don't ask about the whimsical name please):

topdir/applepie/..python.modules...
topdir/applepie/tests/test_*.py
topdir/_darcs/current/...all.the.files...
topdir/_darcs/current/applepie/tests/test_*.py

The output of a collectonly on a fresh checkout:

(er, don't take the Junk directory below the wrong way, I put all my
source tarballs and anonymous checkouts in there)

hamtoro:~/Projects/ApplePie/applepie_2 mick$ py.test --collectonly
inserting into sys.path: /Users/mick/Junk/py-dist
======================================================== test process
starts ========================================================testing-mode:
inprocess
executable:   /usr/local/bin/python  (2.4.1-final-0)
using py lib: /Users/mick/Junk/py-dist/py <rev 11775>

<Directory 'applepie_2'>
    <Directory '_darcs'>
        <Directory 'checkpoints'>
        <Directory 'current'>
            <Directory 'applepie'>
                <Directory 'docs'>
                <Directory 'tests'>
                    <Module 'test_docs.py'>
                        <Generator 'test_docs'>
                            <Function '[0]'>
                    <Module 'test_files.py'>
                        <Function 'test_files_docs'>
            <Directory 'diagrams'>
            <Directory 'scripts'>
        <Directory 'inventories'>
        <Directory 'patches'>
        <Directory 'prefs'>
    <Directory 'applepie'>
        <Directory 'docs'>
        <Directory 'tests'>
            <Module 'test_docs.py'>
            <Module 'test_files.py'>
    <Directory 'diagrams'>
    <Directory 'scripts'>

_____________________________________________________________________________________________________________________________________
    def runtraced(self, colitem):
        if self.shouldclose(): 
            raise Exit, "received external close signal" 
    
        outcome = capture = None 
        # XXX capturing output even for collectors? 
        if not self.config.option.nocapture: 
            if isinstance(colitem, py.test.Item) or not
self.config.option.usepdb:
                capture = SimpleOutErrCapture() 
        needfinish = False
        try: 
            self.start(colitem)
            needfinish = True
            try:
                if colitem._stickyfailure: 
                    raise colitem._stickyfailure 
>               outcome = self.run(colitem) 

[/Users/mick/Junk/py-dist/py/test/session.py:75]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def run(self, colitem): 
        if self.config.option.collectonly and isinstance(colitem,
py.test.Item):
            return 
>       res = colitem.run() 

[/Users/mick/Junk/py-dist/py/test/session.py:99]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def run(self): 
        l = []
>       for name in dir(self.obj): 

[/Users/mick/Junk/py-dist/py/test/collect.py:261]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def obj(self): 
        try: 
            return self._obj    
        except AttributeError:
>           self._obj = obj = self.fspath.pyimport() 

[/Users/mick/Junk/py-dist/py/test/collect.py:288]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def pyimport(self, modname=None, ensuresyspath=True): 
        """ return path as an imported python module. 
            if modname is None, look for the containing package 
            and construct an according module name. 
            The module will be put/looked up in sys.modules. 
        """ 
        if not self.check(): 
            raise py.error.ENOENT(self) 
        #print "trying to import", self
        pkgpath = None
        if modname is None: 
            pkgpath = self.pypkgpath()
            if pkgpath is not None: 
                if ensuresyspath: 
                    self._prependsyspath(pkgpath.dirpath())
                pkg = __import__(pkgpath.basename, None, None, [])
E               assert py.path.local(pkg.__file__).relto(pkgpath) 
>               assert ''
                 +  where '' =
local('/Users/mick/Projects/ApplePie/applepie_2/_darcs/current/applepie/__init__.pyc').relto(local('/Users/mick/Projects/ApplePie/applepie_2/applepie'))
                 +    where
local('/Users/mick/Projects/ApplePie/applepie_2/_darcs/current/applepie/__init__.pyc')
= <class 'py.path.local'>('/Users/mick/Projects/ApplePie/applepie_2/_darcs/current/applepie/__init__.pyc')
                 +      where <class 'py.path.local'> = <Module 'py.path'>.local
                 +        where <Module 'py.path'> = py.path
                 +      and  
'/Users/mick/Projects/ApplePie/applepie_2/_darcs/current/applepie/__init__.pyc'
= <module 'applepie' from
'/Users/mick/Projects/ApplePie/applepie_2/_darcs/current/applepie/__init__.pyc'>.__file__

[/Users/mick/Junk/py-dist/py/path/local/local.py:366]
[testcode  : /Users/mick/Projects/ApplePie/applepie_2/applepie/tests/test_docs.py:2147483648]
[modulepath: ]
_____________________________________________________________________________________________________________________________________
    def runtraced(self, colitem):
        if self.shouldclose(): 
            raise Exit, "received external close signal" 
    
        outcome = capture = None 
        # XXX capturing output even for collectors? 
        if not self.config.option.nocapture: 
            if isinstance(colitem, py.test.Item) or not
self.config.option.usepdb:
                capture = SimpleOutErrCapture() 
        needfinish = False
        try: 
            self.start(colitem)
            needfinish = True
            try:
                if colitem._stickyfailure: 
                    raise colitem._stickyfailure 
>               outcome = self.run(colitem) 

[/Users/mick/Junk/py-dist/py/test/session.py:75]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def run(self, colitem): 
        if self.config.option.collectonly and isinstance(colitem,
py.test.Item):
            return 
>       res = colitem.run() 

[/Users/mick/Junk/py-dist/py/test/session.py:99]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def run(self): 
        l = []
>       for name in dir(self.obj): 

[/Users/mick/Junk/py-dist/py/test/collect.py:261]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def obj(self): 
        try: 
            return self._obj    
        except AttributeError:
>           self._obj = obj = self.fspath.pyimport() 

[/Users/mick/Junk/py-dist/py/test/collect.py:288]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def pyimport(self, modname=None, ensuresyspath=True): 
        """ return path as an imported python module. 
            if modname is None, look for the containing package 
            and construct an according module name. 
            The module will be put/looked up in sys.modules. 
        """ 
        if not self.check(): 
            raise py.error.ENOENT(self) 
        #print "trying to import", self
        pkgpath = None
        if modname is None: 
            pkgpath = self.pypkgpath()
            if pkgpath is not None: 
                if ensuresyspath: 
                    self._prependsyspath(pkgpath.dirpath())
                pkg = __import__(pkgpath.basename, None, None, [])
E               assert py.path.local(pkg.__file__).relto(pkgpath) 
>               assert ''
                 +  where '' =
local('/Users/mick/Projects/ApplePie/applepie_2/_darcs/current/applepie/__init__.pyc').relto(local('/Users/mick/Projects/ApplePie/applepie_2/applepie'))
                 +    where
local('/Users/mick/Projects/ApplePie/applepie_2/_darcs/current/applepie/__init__.pyc')
= <class 'py.path.local'>('/Users/mick/Projects/ApplePie/applepie_2/_darcs/current/applepie/__init__.pyc')
                 +      where <class 'py.path.local'> = <Module 'py.path'>.local
                 +        where <Module 'py.path'> = py.path
                 +      and  
'/Users/mick/Projects/ApplePie/applepie_2/_darcs/current/applepie/__init__.pyc'
= <module 'applepie' from
'/Users/mick/Projects/ApplePie/applepie_2/_darcs/current/applepie/__init__.pyc'>.__file__

[/Users/mick/Junk/py-dist/py/path/local/local.py:366]
[testcode  : /Users/mick/Projects/ApplePie/applepie_2/applepie/tests/test_files.py:2147483648]
[modulepath: ]
_____________________________________________________________________________________________________________________________________=============================================
tests finished: 2 failed in 0.91 seconds
=============================================



And specifying my package:

hamtoro:~/Projects/ApplePie/applepie_2 mick$ py.test applepie --collect
inserting into sys.path: /Users/mick/Junk/py-dist
======================================================== test process
starts ========================================================testing-mode:
inprocess
executable:   /usr/local/bin/python  (2.4.1-final-0)
using py lib: /Users/mick/Junk/py-dist/py <rev 11775>

<Directory 'applepie'>
    <Directory 'docs'>
    <Directory 'tests'>
        <Module 'test_docs.py'>
            <Generator 'test_docs'>
                <Function '[0]'>
        <Module 'test_files.py'>
            <Function 'test_files_docs'>

================================================= tests finished:  in
0.37 seconds =================================================


From jan at balster.info  Mon May  2 15:48:25 2005
From: jan at balster.info (Jan Balster)
Date: Mon, 02 May 2005 15:48:25 +0200
Subject: [py-dev] py.test and darcs
In-Reply-To: <50a522ca050502060728371abe@mail.gmail.com>
References: <50a522ca050502060728371abe@mail.gmail.com>
Message-ID: <42762FA9.3050805@balster.info>

Michael Twomey wrote:
> Hi,
> 
> I've been setting up a new pet project using py.test as the testing
> framework, and darcs as the scm. I've hit a problem after my first
> "darcs record" (darcs checkin). Darcs keeps a copy of the checked out
> originals under a _darcs directory in the root of the source tree,
> which causes py.test some confusion during the collect phase, since it
> has duplicate python modules to contend with.
> 
> Ideally I'd like py.test to ignore _darcs (and I think {arch} for GNU
> arch too) out of the box, but is there a way to twist conftest.py's
> arm to make it ignore these directories?

Create a conftest.py with this content:
########################################################################
import py
class ExcludeDarcsDirectory(py.test.collect.Directory):
	def recfilter(self, path):
		return not path.check(fnmatch="_darcs*") and path.check(dotfile=0)

Directory = ExcludeDarcsDirectory
########################################################################

You can find a more sophisticated example of a custom Collector in
py/documentation/conftest.py.

Have a nice day
Jan



From micktwomey at gmail.com  Mon May  2 16:51:01 2005
From: micktwomey at gmail.com (Michael Twomey)
Date: Mon, 2 May 2005 15:51:01 +0100
Subject: [py-dev] py.test and darcs
In-Reply-To: <42762FA9.3050805@balster.info>
References: <50a522ca050502060728371abe@mail.gmail.com>
	<42762FA9.3050805@balster.info>
Message-ID: <50a522ca050502075132d1aedd@mail.gmail.com>

That worked a treat, thanks.

Michael

On 5/2/05, Jan Balster <jan at balster.info> wrote:
> Michael Twomey wrote:
> > Hi,
> >
> > I've been setting up a new pet project using py.test as the testing
> > framework, and darcs as the scm. I've hit a problem after my first
> > "darcs record" (darcs checkin). Darcs keeps a copy of the checked out
> > originals under a _darcs directory in the root of the source tree,
> > which causes py.test some confusion during the collect phase, since it
> > has duplicate python modules to contend with.
> >
> > Ideally I'd like py.test to ignore _darcs (and I think {arch} for GNU
> > arch too) out of the box, but is there a way to twist conftest.py's
> > arm to make it ignore these directories?
> 
> Create a conftest.py with this content:
> ########################################################################
> import py
> class ExcludeDarcsDirectory(py.test.collect.Directory):
>         def recfilter(self, path):
>                 return not path.check(fnmatch="_darcs*") and path.check(dotfile=0)
> 
> Directory = ExcludeDarcsDirectory
> ########################################################################
> 
> You can find a more sophisticated example of a custom Collector in
> py/documentation/conftest.py.
> 
> Have a nice day
> Jan
> 
> _______________________________________________
> py-dev mailing list
> py-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/py-dev
>


From hpk at trillke.net  Mon May  2 18:20:05 2005
From: hpk at trillke.net (holger krekel)
Date: Mon, 2 May 2005 18:20:05 +0200
Subject: [py-dev] py.test and darcs
In-Reply-To: <42762FA9.3050805@balster.info>
References: <50a522ca050502060728371abe@mail.gmail.com>
	<42762FA9.3050805@balster.info>
Message-ID: <20050502162005.GS22996@solar.trillke.net>

Hi Michael, Hi Jan, 

On Mon, May 02, 2005 at 15:48 +0200, Jan Balster wrote:
> Michael Twomey wrote:
> > Hi,
> > 
> > I've been setting up a new pet project using py.test as the testing
> > framework, and darcs as the scm. I've hit a problem after my first
> > "darcs record" (darcs checkin). Darcs keeps a copy of the checked out
> > originals under a _darcs directory in the root of the source tree,
> > which causes py.test some confusion during the collect phase, since it
> > has duplicate python modules to contend with.
> > 
> > Ideally I'd like py.test to ignore _darcs (and I think {arch} for GNU
> > arch too) out of the box, but is there a way to twist conftest.py's
> > arm to make it ignore these directories?
> 
> Create a conftest.py with this content:
> ########################################################################
> import py
> class ExcludeDarcsDirectory(py.test.collect.Directory):
> 	def recfilter(self, path):
> 		return not path.check(fnmatch="_darcs*") and path.check(dotfile=0)
> 
> Directory = ExcludeDarcsDirectory
> ########################################################################

Yes, that's fine, although 

    def recfilter(self, path): 
        if path.basename == '_darcs': 
            return False
        return super(ExcludeDarcsDirectory, self).recfilter(path)

would be a more "defensive programming" style. 

Michael, if you can investigate/double-check that '_darcs' is indeed
commonly used then i am happy to add it to the list of default
ignored directories.

cheers, 

    holger


From micktwomey at gmail.com  Mon May  2 20:24:32 2005
From: micktwomey at gmail.com (Michael Twomey)
Date: Mon, 2 May 2005 19:24:32 +0100
Subject: [py-dev] py.test and darcs
In-Reply-To: <20050502162005.GS22996@solar.trillke.net>
References: <50a522ca050502060728371abe@mail.gmail.com>
	<42762FA9.3050805@balster.info>
	<20050502162005.GS22996@solar.trillke.net>
Message-ID: <50a522ca0505021124564dd898@mail.gmail.com>

On 5/2/05, holger krekel <hpk at trillke.net> wrote:
> 
> Michael, if you can investigate/double-check that '_darcs' is indeed
> commonly used then i am happy to add it to the list of default
> ignored directories.
> 

Yes, I can confirm that darcs uses _darcs to store all it's data
(including a pristine copy of the code). The darcs manual mentions
_darcs frequently, http://www.abridgegame.org/darcs/manual/

GNU arch does the same using a {arch} top level directory. Both could
change their behaviour in the future to store different data or no
pristine copies, but I don't see a situation where we would want
py.test traversing these directories (arch in particular, it keeps
exceptionally large directory trees under {arch}). The arch manual
mentions {arch} too
(http://regexps.srparish.net/tutorial-tla/arch.html).

Excluding _darcs and {arch} should speed up test collection and avoid
nasty clashes with pristines.

A simple test with a single test_foo.py in an arch checkout (with a
fairly large collection of patches and pristines inside {arch}) shows
that adding {arch} to the excluded directories slashes test time from
an average of 1.5 seconds to 0.5 seconds.

cheers,
  Michael


From hpk at trillke.net  Tue May  3 21:11:01 2005
From: hpk at trillke.net (holger krekel)
Date: Tue, 3 May 2005 21:11:01 +0200
Subject: [py-dev] Re: py.test on win32
In-Reply-To: <20050503181201.19052.qmail@web54504.mail.yahoo.com>
References: <20050503181201.19052.qmail@web54504.mail.yahoo.com>
Message-ID: <20050503191101.GE22996@solar.trillke.net>

Hi Grig, 

i hope you don't mind if I finally repost to py-dev now :-) 

On Tue, May 03, 2005 at 11:12 -0700, Grig Gheorghiu wrote:
> > - python setup.py install 
> >   doesn't install any scripts (the ones in py/bin/*) 
> >   while it does that on unix/osx.  What is an approximate 
> >   way to install such cmdline scripts on win32? 
> > 
> I'm attaching a patch for py/misc/_dist.py which adds a bit of
> win32-specific functionality. It edits the "Path" registry key
> (corresponding to the PATH environment variable) and adds
> os.path.join(sysconfig.get_python_lib(), "py", "bin") to it. Then it
> propagates the changes throughout the system and also to the current
> command prompt session. As a result, you can then type:
> 
> pytest -h
> 
> and it works fine.

sounds great! 

> It's a bit strange maybe to new users that they have to type pytest
> instead of py.test. This is because pytest is actually pytest.cmd,
> which calls "python py.test <args>". One solution would be to rename
> py.test to py.test.win32 if we are on a win32 platform, then rename
> pytest.cmd to py.test.cmd, which will call "python py.test.win32". Then
> users will be able to type py.test instead of pytest. What do you
> think? 

This direction makes sense to me. What about making a subdirectory 
'py/bin/win32' which contains (possibly generated) hooks 
for win32.  For example those hooks would contain something like 

    @echo off
    python %~dp0\..\py.test %*

or - to make it fully generic - find a proper %-escape for the basename
to also work for the other scripts in py/bin/.  (That's not too important 
right now but as you may have noticed the current _dist.py mechanism
works for other py-lib style packages, too.)

> Also, please critique the patch from a code standard point of view if
> you think it doesn't fit with the rest of the code.

Looks fine, except for the bare except at the beginning :-) 

If you send me your desired account name privately, i'll set it 
up on codespeak so that you can directly commit.  I'll see 
the changes on the py-svn list and can still correct/comment on it.   

cheers, 

    holger


From lickspittle at gmail.com  Wed May  4 09:44:28 2005
From: lickspittle at gmail.com (Richard)
Date: Wed, 4 May 2005 07:44:28 +0000
Subject: [py-dev] Greenlets
Message-ID: <ed2577510505040044315e1891@mail.gmail.com>

Hi,

I was wondering if there were any slides for the greenlet presentation
at the ACCU 2005 event and if so, can they be made available?

Also, theres a note at the top of the greenlet source which states
that greenlets are not garbage collected.  Is this fixable? Is what is
required to fix this known and if so, can you please describe it?

Thanks :)
Richard.


From hpk at trillke.net  Thu May  5 00:34:21 2005
From: hpk at trillke.net (holger krekel)
Date: Thu, 5 May 2005 00:34:21 +0200
Subject: [py-dev] py.test and darcs
In-Reply-To: <50a522ca0505021124564dd898@mail.gmail.com>
References: <50a522ca050502060728371abe@mail.gmail.com>
	<42762FA9.3050805@balster.info>
	<20050502162005.GS22996@solar.trillke.net>
	<50a522ca0505021124564dd898@mail.gmail.com>
Message-ID: <20050504223421.GV22996@solar.trillke.net>

Hi Michael, 

On Mon, May 02, 2005 at 19:24 +0100, Michael Twomey wrote:
> On 5/2/05, holger krekel <hpk at trillke.net> wrote:
> > 
> > Michael, if you can investigate/double-check that '_darcs' is indeed
> > commonly used then i am happy to add it to the list of default
> > ignored directories.
> > 
> 
> Yes, I can confirm that darcs uses _darcs to store all it's data
> (including a pristine copy of the code). The darcs manual mentions
> _darcs frequently, http://www.abridgegame.org/darcs/manual/
> 
> GNU arch does the same using a {arch} top level directory. 

OK, i added all of CVS, _darcs and {arch} to the list of ignored 
directories.  This makes it even more important that people 
call "return super(MyDirectory, self).recfilter(path)" 
in their (possibly custom) MyDirectory.recfilter customizations
for paths that they don't want to decide about themselves. 

have fun, 

    holger


From arigo at tunes.org  Fri May  6 14:16:05 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 6 May 2005 14:16:05 +0200
Subject: [py-dev] Re: Greenlets
Message-ID: <20050506121605.GA6392@code1.codespeak.net>

Hi,

The greenlet description at the ACCU was a lightning talk only.  Its
content is now in the greenlet documentation, as the motivating example:

    http://codespeak.net/py/current/doc/greenlet.html#example

Greenlets are not garbage collected in the sense that the following kind
of cycle isn't detected and will leak:

    object in your program ----------> greenlet object

    greenlet object ---internal ref---> suspended frame

    suspended frame ---local variable---> object in your program

Only the second link misses a proper C-level tp_visit.  It could be
implemented by looking at the frame chain of the greenlet object and
calling visit() on each frame, thus meaning that the greenlet object
implicitely holds a reference to each of these suspended frames.  This
would probably help in the common case described above.  In reality, the
greenlet object has a reference to a bunch of bytes that have been
copied verbatim from a part of the C stack; while it is probably safe to
assume that this C stack contains one reference to each of the suspended
frame, it might contain more references to other objects, and it's
almost impossible to know that.


A bientot,

Armin.


From ianb at colorstudy.com  Fri May  6 19:10:55 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 06 May 2005 12:10:55 -0500
Subject: [py-dev] Instantiating Test* classes
Message-ID: <427BA51F.3060508@colorstudy.com>

At some point py.test started instantiating all my classes that start 
with Test*.  This is somewhat problematic, as I have lots of support 
classes named TestSomething, which aren't themselves test cases.  It 
wasn't doing this before, though I must admit I haven't run the tests 
for a while so I'm not sure when the change happened.

I can rename all the classes, but of coures I'd rather not.  Is there 
another way of dealing with this?

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org


From hpk at trillke.net  Sat May  7 09:58:05 2005
From: hpk at trillke.net (holger krekel)
Date: Sat, 7 May 2005 09:58:05 +0200
Subject: [py-dev] Instantiating Test* classes
In-Reply-To: <427BA51F.3060508@colorstudy.com>
References: <427BA51F.3060508@colorstudy.com>
Message-ID: <20050507075805.GH22996@solar.trillke.net>

Hi Ian, 

On Fri, May 06, 2005 at 12:10 -0500, Ian Bicking wrote:
> At some point py.test started instantiating all my classes that start 
> with Test*.  This is somewhat problematic, as I have lots of support 
> classes named TestSomething, which aren't themselves test cases.  

you are talking about classes in test_*.py files, i presume. 

> It wasn't doing this before, though I must admit I haven't run the tests 
> for a while so I'm not sure when the change happened.
> 
> I can rename all the classes, but of coures I'd rather not.  Is there 
> another way of dealing with this?

not that i currently can think of.  you might set 
'disabled=True' on class level. 

    holger


From hpk at trillke.net  Mon May  9 11:43:11 2005
From: hpk at trillke.net (holger krekel)
Date: Mon, 9 May 2005 11:43:11 +0200
Subject: [py-dev] test selection/sets - request for comments
Message-ID: <20050509094311.GD5609@solar.trillke.net>

hi folks, 

recently py.test grew some "test selection" support as a
means to restrict the subset of tests one wants to run.  It's
preliminary and somewhat experimental, see 

    http://codespeak.net/py/current/doc/test.html#selecting-tests-by-keyword

Now i plan to change/extend it in the following way: 

    py.test -m PATTERN  

would select all tests where the "test path" matches the given 
pattern (python regexp).  You would use this to select out a 
specific test function or method that you are currently working
on.  Setup/teardown semantics continue to work as expected, of 
course. 

But for classifying and handling "sets of tests" it's nicer 
to have some explicit keyword-tagging and selection scheme. 

    py.test -k "keyword1 -keyword2" 

would select (it does currently) tests that have keyword1 but
not keyword2, think google syntax.  In PyPy we are already utilizing
the latter form by specifying keywords like 'core' for running
CPython regression tests tagged with the 'core' keyword.  See 

    http://codespeak.net/~hpk/pypy-testresult/

for what we make out of this (we distributedly checkin test 
reports which get evaluated at a central site). 

Currently, it's only possible to associate keywords with tests
by customizing the collection process in a conftest.py file
(which is probably neccessary at least for projects like PyPy
that deeply change the meaning of modules as they 
are loaded and run through the PyPy interpreter).  

However, i am very interested in suggestions on how keywords
should be associated with tests more directly.  My current
thinking (comments welcome!): 

    class TestClass: 
        keywords = "core" 

which would tag all test methods and/or 

    def test_method(): 
        ... 
    test_method.keywords = "core" 

which would just tag one method and/or: 

    keywords = "core network ..." 

which would obviously tag all tests in that module accordingly. 

One usage idea is that one could tag tests with e.g. a
"pendingbug" keyword and would not normally run that unless
you explicitely cared for pending bugs.  This also requires a
way to specify a default set of keywords so that you could set
the default set of to-be-run-keywords to e.g. "-pendingbug" and would
thus not run pending bug tests by default.  Of course, there is
nothing holding you from using bugnumbers like "bug123" that
relate to some issue tracking system. 

Obviously to nail down the exact semantics and coming
up with an easy-to-explain usage/user-level API is not 
completely trivial.  So please share your thoughts and 
usage-ideas (and don't worry about the implementation
too much :-) 

As a bonus task you could come up with a way to specify 
doctests and associating keywords with them :-) 

cheers & thanks, 

    holger

P.S.: all selection/tagging would first be done at 
      collection runtime but for the future we can think about 
      introducing cache file(s) that knows about 
      all test names and keywords so that selecting
      tests by PATTERN or keyword basically becomes 
      an O(1) operation. 


From jan at balster.info  Mon May  9 16:37:53 2005
From: jan at balster.info (Jan Balster)
Date: Mon, 09 May 2005 16:37:53 +0200
Subject: [py-dev] Re: [py-svn] r12006 - in py/dist/py/test: . terminal
In-Reply-To: <20050506103657.15EF427B5D@code1.codespeak.net>
References: <20050506103657.15EF427B5D@code1.codespeak.net>
Message-ID: <427F75C1.2000009@balster.info>

hpk at codespeak.net wrote:
> Author: hpk
> Date: Fri May  6 12:36:56 2005
> New Revision: 12006
> 
> Modified:
>    py/dist/py/test/collect.py
>    py/dist/py/test/item.py
>    py/dist/py/test/terminal/remote.py
> Log:
> make building the collector's tree more persistent 
> by introducing "buildname2items()" which is responsible
> for building a dictionary mapping names to collectors/items. 
> the default run()/join() methods now build this dictionary
> and cache it. 
> 
> The net result is that py lib's own and PyPy's test runs
> run some 10% faster and are somewhat saner!
> 

Hi Holger,

I have a problem with this change. (Test)Items generated by a generator
collector are not executed in the expected order.
Can you change that?

Jan

########################################################################
TestCase:
########################################################################

test_counter = -1
test_list = []

def inc_counter_and_test(expected_number):
    global test_counter
    test_counter += 1
    test_list.append(expected_number)
    counter = test_counter
    assert counter == expected_number

def order_of_execution(expected_list):
    t = test_list
    print 'expected', expected_list
    print 'but got ', t
    assert t == expected_list

def test_generator():
    for i in range(5):
        yield inc_counter_and_test, i
    yield order_of_execution, range(5)

#######################################################################################

Output:
....

    def order_of_execution(expected_list):
        t = test_list
        print 'expected', expected_list
        print 'but got ', t
E       assert t == expected_list
>       assert [4, 2, 3, 0, 1] == [0, 1, 2, 3, 4]

[/.../test_order_of_execution.py:16]
[modulepath: test_generator[5]]
- - - - - - - - - - - - - - - - - - - - - - - - -  [5]: recorded stdout
- - - - - - - - - - - - - - - - - - - - - - - - -
expected [0, 1, 2, 3, 4]
but got  [4, 2, 3, 0, 1]



From hpk at trillke.net  Mon May  9 17:55:32 2005
From: hpk at trillke.net (holger krekel)
Date: Mon, 9 May 2005 17:55:32 +0200
Subject: [py-dev] Re: [py-svn] r12006 - in py/dist/py/test: . terminal
In-Reply-To: <427F75C1.2000009@balster.info>
References: <20050506103657.15EF427B5D@code1.codespeak.net>
	<427F75C1.2000009@balster.info>
Message-ID: <20050509155532.GN5609@solar.trillke.net>

On Mon, May 09, 2005 at 16:37 +0200, Jan Balster wrote:
> hpk at codespeak.net wrote:
> > Author: hpk
> > Date: Fri May  6 12:36:56 2005
> > New Revision: 12006
> > 
> > Modified:
> >    py/dist/py/test/collect.py
> >    py/dist/py/test/item.py
> >    py/dist/py/test/terminal/remote.py
> > Log:
> > make building the collector's tree more persistent 
> > by introducing "buildname2items()" which is responsible
> > for building a dictionary mapping names to collectors/items. 
> > the default run()/join() methods now build this dictionary
> > and cache it. 
> > 
> > The net result is that py lib's own and PyPy's test runs
> > run some 10% faster and are somewhat saner!
> > 
> 
> Hi Holger,
> 
> I have a problem with this change. (Test)Items generated by a generator
> collector are not executed in the expected order.
> Can you change that?

not before the end of the week, sorry (i am just leaving the door). 
can you provide an example meanwhile? 

thanks, 

    holger


From grig at gheorghiu.net  Mon May  9 20:51:32 2005
From: grig at gheorghiu.net (Grig Gheorghiu)
Date: Mon, 9 May 2005 11:51:32 -0700 (PDT)
Subject: [py-dev] Getting 'close failed: [Errno 9] Bad file descriptor' in
	several tests
Message-ID: <20050509185132.31236.qmail@web54505.mail.yahoo.com>

I'm trying to run py.test through its own tests on win32. When I run
test_session.py, I get this:

test_session.py[20] ....................

================== tests finished: 20 passed in 4.59 seconds
==================
close failed: [Errno 9] Bad file descriptor
close failed: [Errno 9] Bad file descriptor

If I disable the last 2 tests in tests_session.py (test_exec and
test_looponfailing), the 'close failed' errors go away.

I get the same exact errors on a Linux RH 9 box, so this seems to be a
problem that is not specific to win32. 

Also, on Linux, when I run the test_gateway.py test suite in
py/execnet, I get:

test_gateway.py[68]
..................................................ssssssssssssssssss


_________________________________________________________________
reasons for skipped tests
_________________________________________________________________
Skipped in /home/ggheo/py/py/execnet/testing/test_gateway.py:287
reason: Skipped: no known ssh target, use -S to set one

=================================================== tests finished: 50
passed, 18 skipped in 3.53 seconds
===================================================
close failed: [Errno 9] Bad file descriptor
close failed: [Errno 9] Bad file descriptor
close failed: [Errno 9] Bad file descriptor
close failed: [Errno 9] Bad file descriptor

I'm still trying to trace the code that's causing the error (it seems
to be related to the _local_close method of a Channel object.)

Grig





From jan at balster.info  Tue May 10 10:47:11 2005
From: jan at balster.info (Jan Balster)
Date: Tue, 10 May 2005 10:47:11 +0200
Subject: [py-dev] Re: [py-svn] r12006 - in py/dist/py/test: . terminal
In-Reply-To: <20050509155532.GN5609@solar.trillke.net>
References: <20050506103657.15EF427B5D@code1.codespeak.net>
	<427F75C1.2000009@balster.info>
	<20050509155532.GN5609@solar.trillke.net>
Message-ID: <4280750F.1020500@balster.info>

holger krekel wrote:

> not before the end of the week, sorry (i am just leaving the door). 
> can you provide an example meanwhile? 
> 

Yes, and a simple hack to fix the problem. I think you will find a
cleaner/better solution :-)

The tests are ordered by their sort value, which is (code.path,
code.firstlineno), and they are stored in a dict(), which doesn't
preserve the order items were added. So tests with the same sort value
don't seem to be executed in the expected order, but that depends on the
implementation of dict.


The function assert_order_of_execution is always defined last, because
otherwise you don't get such a nice error message (test_list could be
empty).

--------------
First example:
--------------

The tests (list_append_x) should be executed from 0 to 2, but they are
executed from 2 to 0.

##############################################################

def test_order_of_execution_generator_different_codeline():
    test_list = []
    expected_list = range(3)

    def list_append_2():
        test_list.append(2)

    def list_append_1():
        test_list.append(1)

    def list_append_0():
        test_list.append(0)

    def assert_order_of_execution():
        print 'expected order', expected_list
        print 'got           ', test_list
        assert test_list == expected_list


    yield list_append_0
    yield list_append_1
    yield list_append_2
    yield assert_order_of_execution

##############################################################

Sample output:

    def assert_order_of_execution():
        print 'expected order', expected_list
        print 'got           ', test_list
E       assert test_list == expected_list
>       assert [2, 1, 0] == [0, 1, 2]

[/.../.../test_order_of_execution.py:38]
[modulepath: test_order_of_execution_generator_different_codeline[3]]
- - - - - - - - - - - - - - - - - - - - - - - - -  [3]: recorded stdout
- - - - - - - - - - - - - - - - - - - - - - - - -
expected order [0, 1, 2]
got            [2, 1, 0]


---------------
Second Example:
---------------

This test only works with len(expected_list) > 3, on Linux with Python
2.4 and Python 2.3.4. It depends on the implementation of dict.

####################################################################
def test_order_of_execution_generator_same_codeline():
    test_list = []
    expected_list = range(6)

    def list_append(item):
        test_list.append(item)

    def assert_order_of_execution():
        print 'expected order', expected_list
        print 'got           ', test_list
        assert test_list == expected_list

    for i in expected_list:
        #test_list.append will raise a TypeError
        #yield lambda x, l = test_list: l.append(x), i
        yield list_append, i
    yield assert_order_of_execution

####################################################################


Sample output:

   def assert_order_of_execution():
        print 'expected order', expected_list
        print 'got           ', test_list
E       assert test_list == expected_list
>       assert [4, 5, 2, 3, 0, 1] == [0, 1, 2, 3, 4, 5]

[/.../.../test_order_of_execution.py:12]
[modulepath: test_order_of_execution_generator_same_codeline[6]]
- - - - - - - - - - - - - - - - - - - - - - - - -  [6]: recorded stdout
- - - - - - - - - - - - - - - - - - - - - - - - -
expected order [0, 1, 2, 3, 4, 5]
got            [4, 5, 2, 3, 0, 1]



-----
Fix:
-----

Index: collect.py
===================================================================
--- collect.py  (revision 12142)
+++ collect.py  (working copy)
@@ -347,7 +347,7 @@
             if not callable(call):
                 raise TypeError("yielded test %r not callable" %(call,))
             name = "[%d]" % i
-            d[name] = self.Function(name, self, args, obj=call)
+            d[name] = self.Function(name, self, args, obj=call,
sort_value = i)
         return d

     def getcallargs(self, obj):



Index: item.py
===================================================================
--- item.py     (revision 12142)
+++ item.py     (working copy)
@@ -45,12 +45,13 @@
         and executing a Python callable test object.
     """
     state = SetupState()
-    def __init__(self, name, parent, args=(), obj=_dummy):
+    def __init__(self, name, parent, args=(), obj=_dummy, sort_value =
None):
         super(Function, self).__init__(name, parent)
         self.args = args
         if obj is not _dummy:
             self._obj = obj
-
+        self.sort_value = sort_value
+
     def __repr__(self):
         return "<%s %r>" %(self.__class__.__name__, self.name)

@@ -59,7 +60,9 @@
         return code.path, code.firstlineno

     def getsortvalue(self):
-        return self.getpathlineno()
+        if self.sort_value is None:
+            return self.getpathlineno()
+        return self.sort_value

     def run(self):
         self.state.prepare(self)











































From ianb at colorstudy.com  Tue May 10 18:34:00 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 10 May 2005 11:34:00 -0500
Subject: [py-dev] Instantiating Test* classes
In-Reply-To: <20050507075805.GH22996@solar.trillke.net>
References: <427BA51F.3060508@colorstudy.com>
	<20050507075805.GH22996@solar.trillke.net>
Message-ID: <4280E278.3040300@colorstudy.com>

holger krekel wrote:
>>At some point py.test started instantiating all my classes that start 
>>with Test*.  This is somewhat problematic, as I have lots of support 
>>classes named TestSomething, which aren't themselves test cases.  
> 
> 
> you are talking about classes in test_*.py files, i presume. 

Yes

>>It wasn't doing this before, though I must admit I haven't run the tests 
>>for a while so I'm not sure when the change happened.
>>
>>I can rename all the classes, but of coures I'd rather not.  Is there 
>>another way of dealing with this?
> 
> 
> not that i currently can think of.  you might set 
> 'disabled=True' on class level. 

That makes sense, but it confuses me that all the tests used to work.  I 
guess before the collect branch was remerged.  I think I'll look into a 
conftest.py file that specifically avoids anything that subclasses 
SQLObject; I have to figure out that stuff anyway.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org


From ianb at colorstudy.com  Tue May 10 19:09:21 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 10 May 2005 12:09:21 -0500
Subject: [py-dev] Instantiating Test* classes
In-Reply-To: <4280E278.3040300@colorstudy.com>
References: <427BA51F.3060508@colorstudy.com>	<20050507075805.GH22996@solar.trillke.net>
	<4280E278.3040300@colorstudy.com>
Message-ID: <4280EAC1.8090401@colorstudy.com>

Ian Bicking wrote:
> That makes sense, but it confuses me that all the tests used to work.  I 
> guess before the collect branch was remerged.  I think I'll look into a 
> conftest.py file that specifically avoids anything that subclasses 
> SQLObject; I have to figure out that stuff anyway.

Here's what I added to sqlobject.conftest that seems to work:

class SQLObjectClass(py.test.collect.Class):
     def run(self):
         if (isinstance(self.obj, type)
             and issubclass(self.obj, sqlobject.SQLObject)):
             return []
         return super(SQLObjectClass, self).run()

# I assume this name magically overrides the normal class collector:
Class = SQLObjectClass

I also clearly have to figure out py.documentation.conftest, as that 
seems to have doctest and ReST hooks that I'll want to use.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org


From ianb at colorstudy.com  Wed May 11 01:04:35 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 10 May 2005 18:04:35 -0500
Subject: [py-dev] Integrating doctests
Message-ID: <42813E03.4010009@colorstudy.com>

OK, so I really want to integrate doctests, along the lines I mention in
this post:

http://blog.ianbicking.org/building-testability-into-web-applications.html

So, here's what I have so far.  There's a couple parts.  Here's a
function to test unit tests:

def unittest_tester(test):
     result = test.defaultTestResult()
     test.run(result)
     for flavor, lst in [('FAIL', result.failures),
                         ('ERROR', result.errors)]:
         for test, err in lst:
             print '%s: %s' % (flavor, test)
             print err
     if result.failures or result.errors:
         py.test.fail('Errors occurred')


This is a little lame, but I haven't gotten far enough into py.test to
figure out reporting.  Actually, I still feel totally mystified by
py.test, I just wander around poking things until they stop breaking.
Is there anyplace I should be starting at to figure out quite what is
going on?  It's kind of that Ravioli Code kind of feel; a description of
the role of all the classes would be really useful.  Though personally I
like descriptions that follow the code path chronologically, it's a good
compliment to the source which tends not to have many chronological hints.

Anyway, I needed that function because I'm using doctest's
TestSuite-building capability, though it'd probably be just as easy to
use doctest.DocTest directly.  Anyway, I really think unittest testing
is a useful feature that belong in there somewhere, in addition to
doctests, since I don't always have the ability to change other
projects' test structure.  I don't think the support has to be very
sophisticated; fix the errors in that example and it might be good
enough.  If it doesn't integrate particularly well into other UIs, like
Tkinter, I don't think that's a big deal.

After that I have this stuff that traverses all modules and gets
doctests from some of the modules:

class Directory(py.test.collect.Directory):
     def filefilter(self, path):
         return (path.check(fnmatch="*.py")
                 and path.purebasename != 'conftest')

class Module(py.test.collect.Module):

     def buildname2items(self):
         if (self.name.startswith('test_')
             or self.name.endswith('_test')):
             d = super(Module, self).buildname2items()
         else:
             d = {}
         try:
             test_suite = doctest.DocTestSuite(
                 self.obj, setUp=setup_doctest,
                 optionflags=flags)
         except ValueError, e:
             if 'has no test' in str(e):
                 return d
             else:
                 raise
         for i, x in py.builtin.enumerate(unpack_testsuite(test_suite)):
             name = 'doctest[%d]' % i
             d[name] = self.Function(name, self, (x,), obj=unittest_tester)
         return d


But this is kind of awkward -- I don't want to look for normal tests in
non-test_*-named files, only doctests, so I have to undo the filefilter
in Directory and then reapply it in Module.  And I get lots of modules
that have no tests in my reports, which wastes space.

So here's what I'd like to do, at which point I'd be fully happy:

* Get rid of spurious tracebacks; the unittest/doctest reporting should
override any py.test reporting entirely.  And of course reports
shouldn't be printed to stdout.  I've been thinking about making things
like doctest; simple input/output tests, except ones that don't occur in
a Python interepreter context, and this would be nice there as well.  I
figured subclassing py.test.item.Item would help somehow, but I don't
know how.  But I'd be quite happy if there was some
py.test.fail_report() function that printed out the message and no
traceback.  That seems easy to use and explain.

* A way to indicate that a module is boring and shouldn't be included,
after the module is loaded and is being collected.  Or maybe empty
modules should just be hidden always.  I don't think I have verbosity
turned up, but maybe I accidentally do and it wouldn't show it normally.

* And of course for unittesting and doctesting to go in py.test
directly, so I don't have complex conftest.py files.  But if that
happens it's really important to me personally that I have some control
over how doctests are made -- adding a setup function is important, and
overriding the option flags (I always want ELLIPSES on, for instance).
I'm trying to avoid boilerplate in the files that are being doctested,
so I'd like to apply these on a package level (i.e., in conftest).  This
seems easy enough to facilitate with a subclass of a to-be-written
DocTestCollector.

While I remember:

* I'd like a way to halt all tests.  With SQLObject your environment
must be configured properly, since you have to access a database.  It's
overwhelming when this isn't the case, because you get a huge list of
failing tests.  I'd like to raise an error when that happens so that
py.test will abort all future tests.  Hmm... I haven't tried SystemExit
or KeyboardInterrupt.  Maybe that'd do on its own.


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org


From grig at gheorghiu.net  Wed May 11 01:05:43 2005
From: grig at gheorghiu.net (Grig Gheorghiu)
Date: Tue, 10 May 2005 16:05:43 -0700 (PDT)
Subject: [py-dev] Intermittent errors on py/misc/testing/test_cache.py
Message-ID: <20050510230543.98594.qmail@web54502.mail.yahoo.com>

I sometimes get this error on a WinXP box:

    def test_cache_eviction(self):
        self.cache.getorbuild(17, lambda: 17)
        py.std.time.sleep(self.maxsecs*1.3)
E       assert self.cache.getentry(17) is None
>       AssertionError

[C:\py\py\misc\testing\test_cache.py:55]
[modulepath: TestAging().test_cache_eviction]

It happens pretty consistently, but not all the time.

I haven't seen it yet on Win2K Pro or Red Hat 9.

Grig



From ianb at colorstudy.com  Wed May 11 01:13:29 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 10 May 2005 18:13:29 -0500
Subject: [py-dev] Tkinter and stdout
Message-ID: <42814019.6090007@colorstudy.com>

I can't see a way to get at the captured stdout through the Tkinter 
interface...?

And incidentally, though I know I'm unusually since I run a funny window 
manager (Ion), it would be nice if the traceback was in a different pane 
in the same window -- in Ion the new window totally obscures the 
previous one, but then in any window manager this sort of thing is a 
problem.  Ideally you'd click on an error, and the bottom pane would 
have tab to display the traceback and stdout (the stdout tab shaded if 
there was none).

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org


From grig at gheorghiu.net  Wed May 11 01:57:05 2005
From: grig at gheorghiu.net (Grig Gheorghiu)
Date: Tue, 10 May 2005 16:57:05 -0700 (PDT)
Subject: [py-dev] Bug tracking
Message-ID: <20050510235705.98759.qmail@web54501.mail.yahoo.com>

Is there a bug tracking system for PyPy set up somewhere other than the
"Issues" at <http://codespeak.net/issues/pypy/> ? I only see 2 issues
filed there, so maybe there's another place for opening bugs and
tracking them.

I'm running py.test on the whole py/py directory structure on win32 and
I'm finding stuff that I'd like to log. Or maybe it's overkill and
sending messages to the py-dev list is enough? Let me know.

Grig


From arigo at tunes.org  Wed May 11 12:10:35 2005
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 11 May 2005 12:10:35 +0200
Subject: [py-dev] Bug tracking
In-Reply-To: <20050510235705.98759.qmail@web54501.mail.yahoo.com>
References: <20050510235705.98759.qmail@web54501.mail.yahoo.com>
Message-ID: <20050511101035.GA9936@code1.codespeak.net>

Hi Grig,

On Tue, May 10, 2005 at 04:57:05PM -0700, Grig Gheorghiu wrote:
> I'm running py.test on the whole py/py directory structure on win32 and
> I'm finding stuff that I'd like to log. Or maybe it's overkill and
> sending messages to the py-dev list is enough? Let me know.

Don't confuse the py lib with PyPy: the py lib is the library containing
py.test, py.path, py.execnet, plus some misc tools.  PyPy is the Python
interpreter written in Python.  It uses the py lib at places for
convenience, and its tests are in py.test format.  The py-dev mailing
list is for the py lib, and the pypy-dev mailing list for PyPy...

Neither project consistently use an issue tracker at the moment, though
Holger is working on changing that soon.  For now I guess you can simply
report problems in the corresponding mailing list.


A bientot,

Armin


From hpk at trillke.net  Thu May 12 13:46:16 2005
From: hpk at trillke.net (holger krekel)
Date: Thu, 12 May 2005 13:46:16 +0200
Subject: [py-dev] Integrating doctests
In-Reply-To: <42813E03.4010009@colorstudy.com>
References: <42813E03.4010009@colorstudy.com>
Message-ID: <20050512114616.GT5609@solar.trillke.net>

Hi Ian, 

i have been away for a few days (and everyone seems to know this
and mails me like crazy).   I have the impression you might have
want to split your mail into multiple ones, but maybe i am just 
missing the red line somehow. 

On Tue, May 10, 2005 at 18:04 -0500, Ian Bicking wrote:
> OK, so I really want to integrate doctests, along the lines I mention in
> this post:
> 
> http://blog.ianbicking.org/building-testability-into-web-applications.html

Can't reach that server at the moment (not the first time, btw). 

> So, here's what I have so far.  There's a couple parts.  Here's a
> function to test unit tests:

You started off with doctests, and now talk about unittests. Probably
your weblog entry would help understand me your line of thinking better ... 

> def unittest_tester(test):
>     result = test.defaultTestResult()
>     test.run(result)
>     for flavor, lst in [('FAIL', result.failures),
>                         ('ERROR', result.errors)]:
>         for test, err in lst:
>             print '%s: %s' % (flavor, test)
>             print err
>     if result.failures or result.errors:
>         py.test.fail('Errors occurred')
> 
> This is a little lame, but I haven't gotten far enough into py.test to
> figure out reporting.

There is definitely a missing py.test customization hook which
allows a user to customize reporting from e.g. a conftest.py
file or from simply invoking py.test.fail()

> Actually, I still feel totally mystified by py.test, I just
> wander around poking things until they stop breaking.  Is
> there anyplace I should be starting at to figure out quite
> what is going on?

I at least wrote some documentation about the internal workings: 

http://codespeak.net/py/current/doc/test.html#collecting-and-running-tests-implementation-remarks

(sorry for the long URL).  Have you read that and deemed it unworthy
or unhelpful? 

> It's kind of that Ravioli Code kind of feel; a description
> of the role of all the classes would be really useful.
> Though personally I like descriptions that follow the code
> path chronologically, it's a good compliment to the source
> which tends not to have many chronological hints.

I had the impression that i keep py-dev updated about refactorings
and the intentions and even provide some detail of what's going 
on.  But that can certainly be improved. 

> Anyway, I needed that function because I'm using doctest's
> TestSuite-building capability, though it'd probably be just as easy to
> use doctest.DocTest directly.  Anyway, I really think unittest testing
> is a useful feature that belong in there somewhere, 

You are mixing unittests and doctests freely here.  I admit i
still haven't looked to closely into doctest's internal workings 
so do you mean to imply that doctest and unittests are very
uniformly handled with Python already? 

> ...  in addition to doctests, since I don't always have the
> ability to change other projects' test structure.

so here you mean that py.test should grok existing
(unit/doc-)test structures?  I am not against the idea but it
is a long way to follow that road i am afraid and i am not
ready to follow that myself at the moment. (apart from the
fact that we have certain ways of running unittests/doctests
in PyPy with py.test mechanisms but it's code that i'd rather
not like to explain at the moment, because it jumps through
more hoops than neccessary: it additionally runs against 
the PyPy interpreter instead of against CPython). 

> I don't think the support has to be very sophisticated; fix
> the errors in that example and it might be good enough.  If
> it doesn't integrate particularly well into other UIs, like
> Tkinter, I don't think that's a big deal.

I'd like Jan Balster's tkinter frontend to remain as much
usable as possible and although i break Jan's code from
time to time (sorry, Jan!) i'd like it very much to be
an alternative frontend. 

> After that I have this stuff that traverses all modules and gets
> doctests from some of the modules:
> 
> class Directory(py.test.collect.Directory):
>     def filefilter(self, path):
>         return (path.check(fnmatch="*.py")
>                 and path.purebasename != 'conftest')
> 
> class Module(py.test.collect.Module):
> 
>     def buildname2items(self):
>         if (self.name.startswith('test_')
>             or self.name.endswith('_test')):
>             d = super(Module, self).buildname2items()
>         else:
>             d = {}
>         try:
>             test_suite = doctest.DocTestSuite(
>                 self.obj, setUp=setup_doctest,
>                 optionflags=flags)
>         except ValueError, e:
>             if 'has no test' in str(e):
>                 return d
>             else:
>                 raise
>         for i, x in py.builtin.enumerate(unpack_testsuite(test_suite)):
>             name = 'doctest[%d]' % i
>             d[name] = self.Function(name, self, (x,), obj=unittest_tester)
>         return d
> 
> 
> But this is kind of awkward -- I don't want to look for normal tests in
> non-test_*-named files, only doctests, so I have to undo the filefilter
> in Directory and then reapply it in Module.  And I get lots of modules
> that have no tests in my reports, which wastes space.

Right.  A better way would be to override the Directory collector 
and produce two kinds of Modules, usual py.test ones and ones 
with doctests with code like this (untested): 

    class Directory(py.test.collect.Directory): 
        def buildname2items(self): 
            # just let our base class do its building
            d = super(Directory, self).buildname2items()

            # let's look for doctests ... 
            for fn in self.fspath.listdir('*.py'): 
                if fn.basename in d or fn.basename == 'conftest.py': 
                    continue
                # we have a candidate for doctests 
                if not isdoctestmodule(fn): # if possible to implement that
                    continue
                d[fn.basename] = DoctestModule(fn, parent=self) 

    class DoctestModule(py.test.collect.Module): 
        def run(self): 
            module = self.fspath.pyimport()
            # setup module appropriately 
            try: 
                # run doctests from 'module'
            finally: 
                # teardown module appropriately 

Like said earlier, this DoctestModule should have a way to "take over" 
reporting somehow. It doesn't at the moment and be warned that there 
might be other slight issues.   But basically the above should  
work rather cleanly. 
            
> * I'd like a way to halt all tests.  With SQLObject your environment
> must be configured properly, since you have to access a database.  It's
> overwhelming when this isn't the case, because you get a huge list of
> failing tests.  I'd like to raise an error when that happens so that
> py.test will abort all future tests.  Hmm... I haven't tried SystemExit
> or KeyboardInterrupt.  Maybe that'd do on its own.

Have you tried py.test.exit(msg)? 

cheers, 

    holger


P.S.: Please note that py.test is still under development and there are 
      sometimes reasons why things are not done yet: often i prefer to 
      refactor and consolidate the (not so small number of) features 
      before i head off adding yet more features.


From hpk at trillke.net  Thu May 12 13:54:50 2005
From: hpk at trillke.net (holger krekel)
Date: Thu, 12 May 2005 13:54:50 +0200
Subject: [py-dev] Intermittent errors on py/misc/testing/test_cache.py
In-Reply-To: <20050510230543.98594.qmail@web54502.mail.yahoo.com>
References: <20050510230543.98594.qmail@web54502.mail.yahoo.com>
Message-ID: <20050512115450.GU5609@solar.trillke.net>

Hi Grig, 

On Tue, May 10, 2005 at 16:05 -0700, Grig Gheorghiu wrote:
> I sometimes get this error on a WinXP box:
> 
>     def test_cache_eviction(self):
>         self.cache.getorbuild(17, lambda: 17)
>         py.std.time.sleep(self.maxsecs*1.3)
> E       assert self.cache.getentry(17) is None
> >       AssertionError

side note: weren't there more details shown in the
next line? 

> [C:\py\py\misc\testing\test_cache.py:55]
> [modulepath: TestAging().test_cache_eviction]
> 
> It happens pretty consistently, but not all the time.
> I haven't seen it yet on Win2K Pro or Red Hat 9.

The test is badly written on my side. Waiting fixed amounts of
time is a bad idea and i am guessing that the time resolution
has interactions with platform details.  I have rewritten the
test to loop longer ... although i think the real solution
would be to set the time that the 'cache' perceives  manually
from the test and then check that the eviction logic does
its work.

cheers, 

    holger


From hpk at trillke.net  Thu May 12 14:12:52 2005
From: hpk at trillke.net (holger krekel)
Date: Thu, 12 May 2005 14:12:52 +0200
Subject: [py-dev] Re: [py-svn] r12006 - in py/dist/py/test: . terminal
In-Reply-To: <4280750F.1020500@balster.info>
References: <20050506103657.15EF427B5D@code1.codespeak.net>
	<427F75C1.2000009@balster.info>
	<20050509155532.GN5609@solar.trillke.net>
	<4280750F.1020500@balster.info>
Message-ID: <20050512121251.GW5609@solar.trillke.net>

Hi Jan! 

On Tue, May 10, 2005 at 10:47 +0200, Jan Balster wrote:
> holger krekel wrote:
> 
> > not before the end of the week, sorry (i am just leaving the door). 
> > can you provide an example meanwhile? 
> > 
> 
> Yes, and a simple hack to fix the problem. I think you will find a
> cleaner/better solution :-)

Thanks for the examples.  What you state and propose all 
makes sense. 

Actually I was thinking about making buildname2items()
building a list so that there would be no sort issues.  I.e. 

    builditems()  -> returns list of items in appropriate 
                     sort order already and/or we simply expect 
                     items to have a sensible __cmp__ method 
                     so that listofitems.sort() always works 
                     as expected. 

and Collector.run() would usually simply do

    self._builditems_if_not_done_already() 
    return [x.name for x in self.items] 
   
and for the join() method we might want to cache a name2items
dictionary mapping if that proves to be a bottleneck.  

What do you think? 

cheers, 

    holger


From jan at balster.info  Thu May 12 17:15:49 2005
From: jan at balster.info (Jan Balster)
Date: Thu, 12 May 2005 17:15:49 +0200
Subject: [py-dev] Capture of stdout
Message-ID: <42837325.3090304@balster.info>

Hi Holger,

is there a simple way to get the captured output before
SimpleOutErrCapture.reset() is called?
Can we add a method for that?

Why would such a function be useful?

Example:
	Contents of TestModule
		
		print 'before test_function'
		
		def test_function():
			print 'test_function'			
			py.test.fail()			

		print 'after test_function'

If a Session instance can get the output of the parent collectors, the
report of test_function would be something like:

....

- - - - -  TestModule: recorded stdout - - - - - - - - -
before test_function

- - - - -  test_function: recorded stdout - - - - - - - - -
test_function


TerminalSession would print all output of the module, not just the part
before test_function was called.
In the current implementation of Terminal it's not necessary to fetch
the output 'in time', because all processing is done at the end.

I would prefer to get the output up to the moment when a test fails. Not
everything a parent collector prints. That's my opinion, and I don't
know if this feature is used in pypy testing.


Jan


From ianb at colorstudy.com  Thu May 12 20:25:45 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 12 May 2005 13:25:45 -0500
Subject: [py-dev] Integrating doctests
In-Reply-To: <20050512114616.GT5609@solar.trillke.net>
References: <42813E03.4010009@colorstudy.com>
	<20050512114616.GT5609@solar.trillke.net>
Message-ID: <42839FA9.2050405@colorstudy.com>

holger krekel wrote:
> Hi Ian, 
> 
> i have been away for a few days (and everyone seems to know this
> and mails me like crazy).   I have the impression you might have
> want to split your mail into multiple ones, but maybe i am just 
> missing the red line somehow. 
> 
> On Tue, May 10, 2005 at 18:04 -0500, Ian Bicking wrote:
> 
>>OK, so I really want to integrate doctests, along the lines I mention in
>>this post:
>>
>>http://blog.ianbicking.org/building-testability-into-web-applications.html
> 
> 
> Can't reach that server at the moment (not the first time, btw). 

Yeah, I know, it sucks :(

> 
>>So, here's what I have so far.  There's a couple parts.  Here's a
>>function to test unit tests:
> 
> 
> You started off with doctests, and now talk about unittests. Probably
> your weblog entry would help understand me your line of thinking better ... 

Sorry.  Doctests can be turned into unittest TestCases (TestSuites 
actaully, but I then turn those into a flat list of cases).  I was using 
that to integrate the doctests.  It's probably better to do it more 
flatly without the levels of wrappers, but the problems are essentially 
the same either way.  The architecture is fairly similar -- 
doctest.DocTest along with doctest.DocTestRunner are like 
unittest.TestCase and/or unittest.TestSuite, with a little 
unittest.TestResult mixed in.

The actual building of unittest test cases from doctests is a little 
crude, so it's probably not the best entry point, but it was expedient 
at the time.

>>def unittest_tester(test):
>>    result = test.defaultTestResult()
>>    test.run(result)
>>    for flavor, lst in [('FAIL', result.failures),
>>                        ('ERROR', result.errors)]:
>>        for test, err in lst:
>>            print '%s: %s' % (flavor, test)
>>            print err
>>    if result.failures or result.errors:
>>        py.test.fail('Errors occurred')
>>
>>This is a little lame, but I haven't gotten far enough into py.test to
>>figure out reporting.
> 
> 
> There is definitely a missing py.test customization hook which
> allows a user to customize reporting from e.g. a conftest.py
> file or from simply invoking py.test.fail()

That would be very useful, and probably solve most of the problems of 
integrating foreign test systems.

>>Actually, I still feel totally mystified by py.test, I just
>>wander around poking things until they stop breaking.  Is
>>there anyplace I should be starting at to figure out quite
>>what is going on?
> 
> 
> I at least wrote some documentation about the internal workings: 
> 
> http://codespeak.net/py/current/doc/test.html#collecting-and-running-tests-implementation-remarks
> 
> (sorry for the long URL).  Have you read that and deemed it unworthy
> or unhelpful? 

I read it a while ago, but I didn't look it up again.  It is helpful. 
There are some internal methods (like buildname2items) that would also 
be useful to have documented here, as they are useful for subclassing.

I was also confused about self.obj -- what is was and where it came 
from.  Especially since there's .obj(), .obj, and ._getobj().  The 
Function properties also confused me.

>>It's kind of that Ravioli Code kind of feel; a description
>>of the role of all the classes would be really useful.
>>Though personally I like descriptions that follow the code
>>path chronologically, it's a good compliment to the source
>>which tends not to have many chronological hints.
> 
> 
> I had the impression that i keep py-dev updated about refactorings
> and the intentions and even provide some detail of what's going 
> on.  But that can certainly be improved. 

I don't mean to complain particularly.  But diving into the code there's 
some parts that threw me.  If I had read the part about the collection 
process just before I did this, that probably would have helped 
considerably, just because I'd know what method to start reading from.

>>Anyway, I needed that function because I'm using doctest's
>>TestSuite-building capability, though it'd probably be just as easy to
>>use doctest.DocTest directly.  Anyway, I really think unittest testing
>>is a useful feature that belong in there somewhere, 
> 
> 
> You are mixing unittests and doctests freely here.  I admit i
> still haven't looked to closely into doctest's internal workings 
> so do you mean to imply that doctest and unittests are very
> uniformly handled with Python already? 
> 
> 
>>...  in addition to doctests, since I don't always have the
>>ability to change other projects' test structure.
> 
> 
> so here you mean that py.test should grok existing
> (unit/doc-)test structures?  I am not against the idea but it
> is a long way to follow that road i am afraid and i am not
> ready to follow that myself at the moment. (apart from the
> fact that we have certain ways of running unittests/doctests
> in PyPy with py.test mechanisms but it's code that i'd rather
> not like to explain at the moment, because it jumps through
> more hoops than neccessary: it additionally runs against 
> the PyPy interpreter instead of against CPython). 

It doesn't seem too hard -- collecting the tests seems easy from what I 
can tell; it's just a matter of doing some isinstance(obj, 
unittest.TestCase) tests, and then this small wrapper to turn test cases 
into py.test Items.  From there it would be nice to add configuration, 
e.g., pass through the verbosity setting, but that's not a core feature. 
  This would simulate most of what unittest.main() does, which isn't a 
whole lot.  To me it seems like it's almost there.  The reporting hook 
would be important.  The collectors we have here are close to good 
enough, at least as a first go.  Is there something I'm missing?  I 
don't think it has to duplicate everyone's custom loaders and whatnot; 
the only valuable part of current tests (IMHO) are the actual TestCases.

>>I don't think the support has to be very sophisticated; fix
>>the errors in that example and it might be good enough.  If
>>it doesn't integrate particularly well into other UIs, like
>>Tkinter, I don't think that's a big deal.
> 
> 
> I'd like Jan Balster's tkinter frontend to remain as much
> usable as possible and although i break Jan's code from
> time to time (sorry, Jan!) i'd like it very much to be
> an alternative frontend. 

By "not particularly well" I meant more "looks like the console embedded 
in a Tkinter window".  If that interface gains new fancy features -- 
which would be great, and I'm also very interested in -- then maybe 
unittest-based tests wouldn't benefit from those features.  But that's 
OK by me.  And maybe doctests would be left behind for a while too, 
which I'd be sad about, but we can fix that up as we go along.

> Right.  A better way would be to override the Directory collector 
> and produce two kinds of Modules, usual py.test ones and ones 
> with doctests with code like this (untested): 
> 
>     class Directory(py.test.collect.Directory): 
>         def buildname2items(self): 
>             # just let our base class do its building
>             d = super(Directory, self).buildname2items()
> 
>             # let's look for doctests ... 
>             for fn in self.fspath.listdir('*.py'): 
>                 if fn.basename in d or fn.basename == 'conftest.py': 
>                     continue
>                 # we have a candidate for doctests 
>                 if not isdoctestmodule(fn): # if possible to implement that
>                     continue
>                 d[fn.basename] = DoctestModule(fn, parent=self) 

I think if you run DocTestFinder on a module you'll get an error or 
empty list.  I suppose you could override DoctestModule.__init__ and 
pass the results from DocTestFinder in.

>     class DoctestModule(py.test.collect.Module): 
>         def run(self): 
>             module = self.fspath.pyimport()
>             # setup module appropriately 
>             try: 
>                 # run doctests from 'module'
>             finally: 
>                 # teardown module appropriately 
> 
> Like said earlier, this DoctestModule should have a way to "take over" 
> reporting somehow. It doesn't at the moment and be warned that there 
> might be other slight issues.   But basically the above should  
> work rather cleanly. 
> 
>>* I'd like a way to halt all tests.  With SQLObject your environment
>>must be configured properly, since you have to access a database.  It's
>>overwhelming when this isn't the case, because you get a huge list of
>>failing tests.  I'd like to raise an error when that happens so that
>>py.test will abort all future tests.  Hmm... I haven't tried SystemExit
>>or KeyboardInterrupt.  Maybe that'd do on its own.
> 
> 
> Have you tried py.test.exit(msg)? 

No I have not, but I will.

> P.S.: Please note that py.test is still under development and there are 
>       sometimes reasons why things are not done yet: often i prefer to 
>       refactor and consolidate the (not so small number of) features 
>       before i head off adding yet more features.

I understand.  I'm just trying to say that almost all the features are 
already there, it's just a couple small things.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org


From jan at balster.info  Thu May 12 20:32:39 2005
From: jan at balster.info (Jan Balster)
Date: Thu, 12 May 2005 20:32:39 +0200
Subject: [py-dev] Re: [py-svn] r12006 - in py/dist/py/test: . terminal
In-Reply-To: <20050512121251.GW5609@solar.trillke.net>
References: <20050506103657.15EF427B5D@code1.codespeak.net>
	<427F75C1.2000009@balster.info>
	<20050509155532.GN5609@solar.trillke.net>
	<4280750F.1020500@balster.info>
	<20050512121251.GW5609@solar.trillke.net>
Message-ID: <4283A147.9030606@balster.info>

holger krekel wrote:
> Hi Jan! 
> 
> On Tue, May 10, 2005 at 10:47 +0200, Jan Balster wrote:
> 
>>holger krekel wrote:
>>
>>
>>>not before the end of the week, sorry (i am just leaving the door). 
>>>can you provide an example meanwhile? 
>>>
>>
>>Yes, and a simple hack to fix the problem. I think you will find a
>>cleaner/better solution :-)
> 
> 
> Thanks for the examples.  What you state and propose all 
> makes sense.

You can find the testfile in my user directory.

> 
> Actually I was thinking about making buildname2items()
> building a list so that there would be no sort issues.  

Sounds good.


Jan








From ianb at colorstudy.com  Fri May 13 00:46:25 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 12 May 2005 17:46:25 -0500
Subject: [py-dev] Integrating doctests
In-Reply-To: <42839FA9.2050405@colorstudy.com>
References: <42813E03.4010009@colorstudy.com>	<20050512114616.GT5609@solar.trillke.net>
	<42839FA9.2050405@colorstudy.com>
Message-ID: <4283DCC1.2030009@colorstudy.com>

Ian Bicking wrote:
>>> OK, so I really want to integrate doctests, along the lines I mention in
>>> this post:
>>>
>>> http://blog.ianbicking.org/building-testability-into-web-applications.html 

Incidentally, if this is down, this should be up:

http://webwareforpython.org/archives/message/20050510.043224.66fea48c.en.html

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org


From ianb at colorstudy.com  Fri May 13 21:00:14 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 13 May 2005 14:00:14 -0500
Subject: [py-dev] Backtracking on doctest
Message-ID: <4284F93E.8030805@colorstudy.com>

Well, now I'm starting to backtrack.  After I extended my test while 
adding new features, I looked back over my doctests and realized they 
were pretty pointless.  For brevity, most of my code no longer produced 
output at all, but instead was doing internal inserts.  If I didn't have 
a wrapper around my application for testing this might not have been 
feasible, but without a wrapper the testing wouldn't be feasible.

Anyway, I converted it to a normal py.test test.  It seems much more 
reasonable.  I'm a little sad that it's not sitting right next to the 
servlet that it tests -- I liked that the tests were in a docstring 
(though they were also becoming rather long-winded, so maybe that's not 
ultimately practical either).  And I also found myself wanting features 
like the assert reinterpretation, which I lost in doctests.

setup_module turned out to be quite useful; I added this to my conftest, 
and do a "from conftest import setup_module" in my module:

def setup_module(module):
     app = TestApp(server.make_app(CONFIG.current_conf()),
                   CONFIG.current_conf())
     module.app = app
     module.CONFIG = CONFIG
     fixture.reset_state()


I think it's likely that other boilerplate stuff will go in there in the 
future.  I'd still like doctests to work nicely, but it becomes less 
important to me.

Anyway, no requests here, just thought I'd share my current and always 
changing thoughts on the matter.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org


From ianb at colorstudy.com  Fri May 13 21:29:54 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 13 May 2005 14:29:54 -0500
Subject: [py-dev] keyword selection
Message-ID: <42850032.5000306@colorstudy.com>

I was a little surprised with the keyword "worse" didn't match the test 
"test_worse_upload".  I'd have expected "test_" to be redundant.  But 
I'd probably expect "upload" to also match that test; I think I'd 
normally expect that it would match "words", with words indicated by _'s 
or mixed case.  And actually I can imagine that for this test I'm 
writing -- I have several tests for CSV uploads, and several for uploads 
in general, and it seems natural to use the keyword "csv" for one and 
"upload" for the other.

But this is just selecting a single test in a larger module; I haven't 
tried it for larger sets of tests, so it might feel more natural there. 
  Either way requiring "test_" seems unnecessary.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org


From hpk at trillke.net  Fri May 13 21:32:54 2005
From: hpk at trillke.net (holger krekel)
Date: Fri, 13 May 2005 21:32:54 +0200
Subject: [py-dev] Backtracking on doctest
In-Reply-To: <4284F93E.8030805@colorstudy.com>
References: <4284F93E.8030805@colorstudy.com>
Message-ID: <20050513193254.GT5609@solar.trillke.net>

On Fri, May 13, 2005 at 14:00 -0500, Ian Bicking wrote:
> Well, now I'm starting to backtrack.  After I extended my test while 
> adding new features, I looked back over my doctests and realized they 
> were pretty pointless.  For brevity, most of my code no longer produced 
> output at all, but instead was doing internal inserts.  If I didn't have 
> a wrapper around my application for testing this might not have been 
> feasible, but without a wrapper the testing wouldn't be feasible.

Interesting.  With PyPy we noticed that we have lots of problems
with some doctest/output-comparison related CPython regression 
tests.  Doctests and output-comparisons are pretty sensitive 
to the exact way things are represented as strings.  Nevertheless
i still like the general doctest idea. 

> Anyway, I converted it to a normal py.test test.  It seems much more 
> reasonable.  I'm a little sad that it's not sitting right next to the 
> servlet that it tests -- I liked that the tests were in a docstring 
> (though they were also becoming rather long-winded, so maybe that's not 
> ultimately practical either).  

Did you consider putting tests in the same source file as the code
being tested?  It's probably often not too nice but i am using it
for one-file scripts and simply invoke 'py.test myscript.py'. 

> setup_module turned out to be quite useful; I added this to my conftest, 
> and do a "from conftest import setup_module" in my module:
> 
> def setup_module(module):
>     app = TestApp(server.make_app(CONFIG.current_conf()),
>                   CONFIG.current_conf())
>     module.app = app
>     module.CONFIG = CONFIG
>     fixture.reset_state()

I guess we should start a "recipe" section in the
documentation for nice ideas like this.    

Btw, i once thought about making your above
conftest.setup_module(module) get invoked automagically just
like Collectors/Items are looked up.  An advantage could be
that you could stack setup_module initializations but i am not
sure how useful that is (it might be for larger projects).
It's easy to add this feature should we want it. 

> I think it's likely that other boilerplate stuff will go in there in the 
> future.  I'd still like doctests to work nicely, but it becomes less 
> important to me.
>
> Anyway, no requests here, just thought I'd share my current and always 
> changing thoughts on the matter.

Thanks a lot.  I admit that your "ravioli" term used earlier deeply 
hurt my feelings until i realized that you didn't say Spaghetti :-) 

cheers, 
    
    holger


From hpk at trillke.net  Fri May 13 21:40:01 2005
From: hpk at trillke.net (holger krekel)
Date: Fri, 13 May 2005 21:40:01 +0200
Subject: [py-dev] keyword selection
In-Reply-To: <42850032.5000306@colorstudy.com>
References: <42850032.5000306@colorstudy.com>
Message-ID: <20050513194001.GU5609@solar.trillke.net>

On Fri, May 13, 2005 at 14:29 -0500, Ian Bicking wrote:
> I was a little surprised with the keyword "worse" didn't match the test 
> "test_worse_upload".  I'd have expected "test_" to be redundant.  But 
> I'd probably expect "upload" to also match that test; I think I'd 
> normally expect that it would match "words", with words indicated by _'s 
> or mixed case.  And actually I can imagine that for this test I'm 
> writing -- I have several tests for CSV uploads, and several for uploads 
> in general, and it seems natural to use the keyword "csv" for one and 
> "upload" for the other.
>
> But this is just selecting a single test in a larger module; I haven't 
> tried it for larger sets of tests, so it might feel more natural there. 
>  Either way requiring "test_" seems unnecessary.

I agree that is quite unsatisfactory. Did you see

    http://codespeak.net/pipermail/py-dev/2005-May/000357.html

where i asked for comments about test selection support?  
I would be most interested in your opinion.  The presented ideas 
should not be too hard to implement but i'd like to get some 
feedback before actually doing it. 

cheers, 

    holger


From grig at gheorghiu.net  Fri May 13 21:49:15 2005
From: grig at gheorghiu.net (Grig Gheorghiu)
Date: Fri, 13 May 2005 12:49:15 -0700 (PDT)
Subject: [py-dev] Getting 'close failed: [Errno 9] Bad file descriptor' in
	several tests
In-Reply-To: 6667
Message-ID: <20050513194915.23413.qmail@web54508.mail.yahoo.com>

Just a heads-up. I'm still looking into this issue -- I got deep in the
bowels of execnet gateways, channels and messages. This may be
caused by some sort of race condition which causes a channel to get
deleted, then to continue to receive data. Also, I always see this
message, which may also be a symptom:

waiting for pid <PID>
child process <PID> already dead? error:[Errno 10] No child processes

Here's an output with debugging messages. The test_session1.py file is
a copy of py/test/testing/test_session.py file, with only the test_exec
test in it.

Note how the channel with id 5 gets deleted at some point, then it
still receives data (I modified a bunch of files in execnet so that I
can print more helpful debugging info):

C:\py\py\test\testing>py.test test_session1.py  -s
inserting into sys.path: C:\py
============================= test process starts
=============================
testing-mode: inprocess
executable:   C:\Python24\python.exe  (2.4.0-final-0)
***** svn info C:\py\py
using py lib: C:\py\py <rev unknown>

test_session1.py[1] C:\Python24\python.exe
sending gateway bootstrap code
Creating channel with id 1
sent -> <Message.CHANNEL_OPEN channelid=1 len=54>
Creating channel with id 3
Creating channel with id 5
Creating channel with id 7
sent -> <Message.CHANNEL_OPEN channelid=3 len=89>
sent -> <Message.CHANNEL_DATA channelid=3 len=114>
received <- <Message.CHANNEL_DATA channelid=1 2960>
Got CHANNEL_DATA for channel 1
received <- <Message.CHANNEL_CLOSE channelid=1 ''>
Got CHANNEL_CLOSE; closing channel 1
deleting channel mapping 1
received <- <Message.CHANNEL_DATA channelid=5 len=83>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 'testing-mode: child
process\n'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 len=59>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 '*****'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 ' '>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 'svn info C:\\py\\py'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 '\n'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 'using py lib: C:\\py\\py
<rev unk
nown>\n'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 '\n'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 'None'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 '[1] '>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 'F'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 '\n'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 '\n'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 len=83>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 '\n'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 '    def test_1():\n'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 'E       assert 1 == 0
\n'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 '>       assert 1 ==
0\n'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 '\n'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 len=91>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 '[modulepath: test_1]\n'>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 len=83>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 len=343>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 len=81>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 len=117>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 len=83>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=5 len=82>
Got CHANNEL_DATA for channel 5
received <- <Message.CHANNEL_DATA channelid=3 len=113>
Got CHANNEL_DATA for channel 3
received <- <Message.CHANNEL_CLOSE channelid=5 ''>
Got CHANNEL_CLOSE; closing channel 5
deleting channel mapping 5
received <- <Message.CHANNEL_DATA channelid=5 'Traceback (most recent
call last)
:\n'>
Got CHANNEL_DATA for channel 5
dict {3: <Channel id=3 open>, 7: <Channel id=7 open>} doesn't have key
5
received <- <Message.CHANNEL_DATA channelid=5 '  File "<string>", line
415, in t
race\n'>
Got CHANNEL_DATA for channel 5
dict {3: <Channel id=3 open>, 7: <Channel id=7 open>} doesn't have key
5
received <- <Message.CHANNEL_DATA channelid=5 'IOError: [Errno 22]
Invalid argum
ent\n'>
Got CHANNEL_DATA for channel 5
dict {3: <Channel id=3 open>, 7: <Channel id=7 open>} doesn't have key
5
received <- <Message.CHANNEL_CLOSE channelid=3 ''>
Got CHANNEL_CLOSE; closing channel 3
deleting channel mapping 3
closing down channel and gateway
acquiring _exitlock
_exitlock acquired
exit procedure triggered, pid 3116
sent -> <Message.EXIT_GATEWAY channelid=0 ''>
leaving <Thread(sender0, started)>
releasing _exitlock
_exitlock released
waiting for pid 2960
child process 2960 already dead? error:[Errno 10] No child processes
.

================== tests finished: 1 passed in 0.58 seconds
==================
====================cleaning up====================
received <- <Message.STOP_RECEIVING channelid=0 ''>
Got STOP_RECEIVING; closing channel <Channel id=7 open>
deleting channel mapping 7
leaving <Thread(receiver0, started)>
close failed: [Errno 9] Bad file descriptor
Grig

--- Grig Gheorghiu <grig at gheorghiu.net> wrote:
> I'm trying to run py.test through its own tests on win32. When I run
> test_session.py, I get this:
> 
> test_session.py[20] ....................
> 
> ================== tests finished: 20 passed in 4.59 seconds
> ==================
> close failed: [Errno 9] Bad file descriptor
> close failed: [Errno 9] Bad file descriptor
> 
> If I disable the last 2 tests in tests_session.py (test_exec and
> test_looponfailing), the 'close failed' errors go away.
> 
> I get the same exact errors on a Linux RH 9 box, so this seems to be
> a
> problem that is not specific to win32. 
> 
> Also, on Linux, when I run the test_gateway.py test suite in
> py/execnet, I get:
> 
> test_gateway.py[68]
> ..................................................ssssssssssssssssss
> 
> 
> _________________________________________________________________
> reasons for skipped tests
> _________________________________________________________________
> Skipped in /home/ggheo/py/py/execnet/testing/test_gateway.py:287
> reason: Skipped: no known ssh target, use -S to set one
> 
> =================================================== tests finished:
> 50
> passed, 18 skipped in 3.53 seconds
> ===================================================
> close failed: [Errno 9] Bad file descriptor
> close failed: [Errno 9] Bad file descriptor
> close failed: [Errno 9] Bad file descriptor
> close failed: [Errno 9] Bad file descriptor
> 
> I'm still trying to trace the code that's causing the error (it seems
> to be related to the _local_close method of a Channel object.)
> 
> Grig
> 
> 
> 
> _______________________________________________
> py-dev mailing list
> py-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/py-dev
> 


From ianb at colorstudy.com  Fri May 13 22:00:16 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 13 May 2005 15:00:16 -0500
Subject: [py-dev] Backtracking on doctest
In-Reply-To: <20050513193254.GT5609@solar.trillke.net>
References: <4284F93E.8030805@colorstudy.com>
	<20050513193254.GT5609@solar.trillke.net>
Message-ID: <42850750.4030605@colorstudy.com>

holger krekel wrote:
> On Fri, May 13, 2005 at 14:00 -0500, Ian Bicking wrote:
> 
>>Well, now I'm starting to backtrack.  After I extended my test while 
>>adding new features, I looked back over my doctests and realized they 
>>were pretty pointless.  For brevity, most of my code no longer produced 
>>output at all, but instead was doing internal inserts.  If I didn't have 
>>a wrapper around my application for testing this might not have been 
>>feasible, but without a wrapper the testing wouldn't be feasible.
> 
> 
> Interesting.  With PyPy we noticed that we have lots of problems
> with some doctest/output-comparison related CPython regression 
> tests.  Doctests and output-comparisons are pretty sensitive 
> to the exact way things are represented as strings.  Nevertheless
> i still like the general doctest idea. 

It's not too terribly hard to override the comparison that doctest does. 
  Personally I like fuzzier comparisons in general.  There's not a very 
good way to install those comparisons, so if py.test integrates doctest 
it would be nice to have a convention for that.

For instance, for HTML-generating code I've written code to do more 
semantic XML comparisons that ignore some whitespace and the order of 
attributes.

>>Anyway, I converted it to a normal py.test test.  It seems much more 
>>reasonable.  I'm a little sad that it's not sitting right next to the 
>>servlet that it tests -- I liked that the tests were in a docstring 
>>(though they were also becoming rather long-winded, so maybe that's not 
>>ultimately practical either).  
> 
> 
> Did you consider putting tests in the same source file as the code
> being tested?  It's probably often not too nice but i am using it
> for one-file scripts and simply invoke 'py.test myscript.py'. 

I had, but I'm not sure if it would help a lot.  I'd still need the 
hooks in conftest, because I'd wanted those tests to be automatically 
picked up.  If it's just a function at the bottom of the module, I don't 
think that's actually much better than a separate module.

I think I had this idea that I could put ten lines of doctest at the top 
of my module that would be this concise documentation and description of 
the module, plus a test.  I'm not sure that's feasible, though it is 
still possible to use that for basic exercising of the code -- basically 
a one-liner that tells what a valid URL would be.  And, maybe, with a 
subclass of the standard Paste fixture you could add things like:

   app.upload_csv([('Name', 'Email'), ('Bob', 'bob at company.com')])

And that could be high enough level to get some benefit from being in a 
docstring.  But those kinds of methods (like "upload_csv") would have to 
be implemented in a class specifically meant for testing a single 
screen; which is also a useful idea, but I haven't really gotten that 
far in generalizing this stuff.

>>setup_module turned out to be quite useful; I added this to my conftest, 
>>and do a "from conftest import setup_module" in my module:
>>
>>def setup_module(module):
>>    app = TestApp(server.make_app(CONFIG.current_conf()),
>>                  CONFIG.current_conf())
>>    module.app = app
>>    module.CONFIG = CONFIG
>>    fixture.reset_state()
> 
> 
> I guess we should start a "recipe" section in the
> documentation for nice ideas like this.    

Once I get this all figured out I'll probably document the entire 
process for Paste, and include this code somewhere in the project.

> Btw, i once thought about making your above
> conftest.setup_module(module) get invoked automagically just
> like Collectors/Items are looked up.  An advantage could be
> that you could stack setup_module initializations but i am not
> sure how useful that is (it might be for larger projects).
> It's easy to add this feature should we want it. 

I could speculate, but I should probably keep trying to implement tests 
and then figure out what's works and what feels awkward.  Since I only 
have one module tested so far, "boilerplate" in that one module is no 
worse than a line of code in conftest ;)

>>I think it's likely that other boilerplate stuff will go in there in the 
>>future.  I'd still like doctests to work nicely, but it becomes less 
>>important to me.
>>
>>Anyway, no requests here, just thought I'd share my current and always 
>>changing thoughts on the matter.
> 
> 
> Thanks a lot.  I admit that your "ravioli" term used earlier deeply 
> hurt my feelings until i realized that you didn't say Spaghetti :-) 

Heh, sorry ;)  It's a term I came upon one day on the wiki 
(http://c2.com/cgi/wiki?RavioliCode) and it kind of stuck in my mind as 
a useful metaphor, but I forget that no one else knows what I mean.  If 
I had read up and knew what method to start looking at, I'd probably 
wouldn't have felt that way -- though it's always a risk with 
dynamically typed code, because in isolation it can be hard to tell what 
object you are interacting with.  For instance, I had a hard time 
figuring out when a variable was the object itself, or its collection 
wrapper (e.g., collect.Module or the actual module object).

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org


From ianb at colorstudy.com  Fri May 13 22:18:11 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 13 May 2005 15:18:11 -0500
Subject: [py-dev] keyword selection
In-Reply-To: <20050513194001.GU5609@solar.trillke.net>
References: <42850032.5000306@colorstudy.com>
	<20050513194001.GU5609@solar.trillke.net>
Message-ID: <42850B83.6030605@colorstudy.com>

holger krekel wrote:
> I agree that is quite unsatisfactory. Did you see
> 
>     http://codespeak.net/pipermail/py-dev/2005-May/000357.html
> 
> where i asked for comments about test selection support?  
> I would be most interested in your opinion.  The presented ideas 
> should not be too hard to implement but i'd like to get some 
> feedback before actually doing it. 

Ah, yes, I saw that go by but didn't have any opinions (I guess I wasn't 
in the right mindset at the time).

> Now i plan to change/extend it in the following way: 
> 
>     py.test -m PATTERN  
> 
> would select all tests where the "test path" matches the given 
> pattern (python regexp).  You would use this to select out a 
> specific test function or method that you are currently working
> on.  Setup/teardown semantics continue to work as expected, of 
> course. 

Generally I dislike regex matching, I'd rather see shell-style path 
matching, like */*upload* -- maybe as an extension ** could match 
multiple "segments", like **/*upload* matches every test with "upload" 
in its name.  This feels very natural to me, whereas the regex 
equivalents -- "[^/]+/[^/]*upload[^/]*", "upload[^/]*" -- hurt my eyes.

> But for classifying and handling "sets of tests" it's nicer 
> to have some explicit keyword-tagging and selection scheme. 
> 
>     py.test -k "keyword1 -keyword2" 
> 
> would select (it does currently) tests that have keyword1 but
> not keyword2, think google syntax.  

I think that's a good way to match.  I assume "keyword1 keyword2" means 
keyword1 AND keyword2?  I can imagine that multiple -k arguments could 
mean OR.
> 
> However, i am very interested in suggestions on how keywords
> should be associated with tests more directly.  My current
> thinking (comments welcome!): 
> 
>     class TestClass: 
>         keywords = "core" 
> 
> which would tag all test methods and/or 
> 
>     def test_method(): 
>         ... 
>     test_method.keywords = "core" 
> 
> which would just tag one method and/or: 
> 
>     keywords = "core network ..." 
> 
> which would obviously tag all tests in that module accordingly. 

I think these seem generally good enough.  But should names themselves 
be keywords?  Maybe only if the keywords weren't specified explicitly.

> As a bonus task you could come up with a way to specify 
> doctests and associating keywords with them :-) 

Well, one nice aspect of doctests is that they ignore a lot, so maybe 
you'd just search for ^\s+keywords\s*=(.*)$ in the docstring.  As to a 
means of indicating that something is a doctest, or that a module should 
be doctested, hrm.  Well, the presence of the variable __test__ is a 
good sign, as doctest uses that for testing (both at the module and 
class level).  But even that requires loading all the modules, and most 
doctestable modules don't have __test__ anyway.  It might be good enough 
to just do some enumeration, probably using wildcards again (e.g., 
examples/**/*.py, docs/*.txt, etc.)

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org


From hpk at trillke.net  Fri May 13 22:33:06 2005
From: hpk at trillke.net (holger krekel)
Date: Fri, 13 May 2005 22:33:06 +0200
Subject: [py-dev] Getting 'close failed: [Errno 9] Bad file descriptor' in
	several tests
In-Reply-To: <20050513194915.23413.qmail@web54508.mail.yahoo.com>
References: <20050513194915.23413.qmail@web54508.mail.yahoo.com>
Message-ID: <20050513203306.GW5609@solar.trillke.net>

Hey Grig, 

at the very minute your mail arrived i wanted to start mailing you :-) 

On Fri, May 13, 2005 at 12:49 -0700, Grig Gheorghiu wrote:
> Just a heads-up. I'm still looking into this issue -- I got deep in the
> bowels of execnet gateways, channels and messages. This may be
> caused by some sort of race condition which causes a channel to get
> deleted, then to continue to receive data.  

Actually i noticed similar problems than the ones you
mentioned earlier on linux platforms, usually while execing
with e.g. '--exec=python2.4'.  There probably is some race
condition and likely it's "just" an unclean shutdown procedure
(shutdown is really quite involved with network code and i
guess the according py.execnet code needs a thorough human
review because it's usually hard to find problems by just
looking at failing or oddly-running tests). 

> Also, I always see this
> message, which may also be a symptom:
> 
> waiting for pid <PID>
> child process <PID> already dead? error:[Errno 10] No child processes

That's probably related to my not knowing how to properly 
wait for a child process to terminate on win32.  If you
don't do a waitpid(childprocessid) on unix then your child
process will become a "zombie".  Is that even a problem on 
win32?  If not, then one might try to just skip this
code for win32?! 

> Here's an output with debugging messages. The test_session1.py file is
> a copy of py/test/testing/test_session.py file, with only the test_exec
> test in it.
>
> Note how the channel with id 5 gets deleted at some point, then it
> still receives data (I modified a bunch of files in execnet so that I
> can print more helpful debugging info):

feel free to commit tracing additions. 

> C:\py\py\test\testing>py.test test_session1.py  -s
> inserting into sys.path: C:\py
> ============================= test process starts
> =============================
> testing-mode: inprocess
> executable:   C:\Python24\python.exe  (2.4.0-final-0)
> ***** svn info C:\py\py
> using py lib: C:\py\py <rev unknown>
> 
> test_session1.py[1] C:\Python24\python.exe
> sending gateway bootstrap code
> Creating channel with id 1
> sent -> <Message.CHANNEL_OPEN channelid=1 len=54>
> Creating channel with id 3
> Creating channel with id 5
> Creating channel with id 7

The last two channels are created for receiving the output
from the redirected sys.stdout/stderr from the remote side. 

> sent -> <Message.CHANNEL_OPEN channelid=3 len=89>
> sent -> <Message.CHANNEL_DATA channelid=3 len=114>
> received <- <Message.CHANNEL_DATA channelid=1 2960>
> Got CHANNEL_DATA for channel 1
> received <- <Message.CHANNEL_CLOSE channelid=1 ''>
> Got CHANNEL_CLOSE; closing channel 1
> deleting channel mapping 1
> received <- <Message.CHANNEL_DATA channelid=5 len=83>
> Got CHANNEL_DATA for channel 5
> received <- <Message.CHANNEL_DATA channelid=5 'testing-mode: child
> process\n'>

So channelid=5 is really responsible for stdout. 

> Got CHANNEL_DATA for channel 5
> received <- <Message.CHANNEL_DATA channelid=5 len=59>
> ... 
> ...
> Got CHANNEL_DATA for channel 3
> received <- <Message.CHANNEL_CLOSE channelid=5 ''>

here the stdout-channel gets closed at the other side ... 

> Got CHANNEL_CLOSE; closing channel 5
> deleting channel mapping 5
> received <- <Message.CHANNEL_DATA channelid=5 'Traceback (most recent
> call last)
> :\n'>

but we still receive stdout-data from the other side.  
This should never happen.  I guess that the channel close-logic 
is not correct in that it deletes the channel mapping too early. 
Or the redirection-per-thread code in py/thread/io.py is not 
working completely.  Or both :-/  

cheers, 

    holger


From michele.simionato at gmail.com  Sat May 14 05:21:19 2005
From: michele.simionato at gmail.com (Michele Simionato)
Date: Fri, 13 May 2005 23:21:19 -0400
Subject: [py-dev] Backtracking on doctest
In-Reply-To: <42850750.4030605@colorstudy.com>
References: <4284F93E.8030805@colorstudy.com>
	<20050513193254.GT5609@solar.trillke.net>
	<42850750.4030605@colorstudy.com>
Message-ID: <4edc17eb05051320217631e28@mail.gmail.com>

On 5/13/05, Ian Bicking <ianb at colorstudy.com> wrote:

> I think I had this idea that I could put ten lines of doctest at the top
> of my module that would be this concise documentation and description of
> the module, plus a test.  I'm not sure that's feasible, though it is
> still possible to use that for basic exercising of the code -- basically
> a one-liner that tells what a valid URL would be.  And, maybe, with a
> subclass of the standard Paste fixture you could add things like:
> 
>    app.upload_csv([('Name', 'Email'), ('Bob', 'bob at company.com')])
> 
> And that could be high enough level to get some benefit from being in a
> docstring.  But those kinds of methods (like "upload_csv") would have to
> be implemented in a class specifically meant for testing a single
> screen; which is also a useful idea, but I haven't really gotten that
> far in generalizing this stuff.

I mostly use this convention: for any mymodule.py file I have a 
mymodule.txt file containing the documentation and the basic
doctests for that module. There may be additional doctests which
do not add value to the documentation, and/or unittests: all this is
put them in a subdirectory called "test".
I tend to put very few or zero tests in the docstring.
I have my own recipe to run the text files

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/410052

and another to run the test suite on the entire package (not yet published).


          Michele Simionato


From micktwomey at gmail.com  Sat May 14 11:31:16 2005
From: micktwomey at gmail.com (Michael Twomey)
Date: Sat, 14 May 2005 10:31:16 +0100
Subject: [py-dev] py.test exit code 0 even on failure
Message-ID: <50a522ca050514023177fd4c58@mail.gmail.com>

Hi,

I've been plugging py.test into darc's automatic test features and
I've found that py.test doesn't appear to exit with anything other
than a 0, even when there are failures. This means that darcs won't
notice that there were test failures on my commits. I haven't verified
that this is the case on anything other than my mac.

The following patch makes py.test exit with a 1 when there are
problems (or at least a list of stuff returned by main). This works
for me, though I'm sure there is a better spot/way to do this.

cheers,
  mick
-------------- next part --------------
A non-text attachment was scrubbed...
Name: py_test_exit.diff
Type: application/octet-stream
Size: 546 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20050514/bc269648/attachment.obj>

From hpk at trillke.net  Sat May 14 12:31:44 2005
From: hpk at trillke.net (holger krekel)
Date: Sat, 14 May 2005 12:31:44 +0200
Subject: [py-dev] py.test exit code 0 even on failure
In-Reply-To: <50a522ca050514023177fd4c58@mail.gmail.com>
References: <50a522ca050514023177fd4c58@mail.gmail.com>
Message-ID: <20050514103144.GE5609@solar.trillke.net>

Hi Michael, 

On Sat, May 14, 2005 at 10:31 +0100, Michael Twomey wrote:
> I've been plugging py.test into darc's automatic test features and
> I've found that py.test doesn't appear to exit with anything other
> than a 0, even when there are failures. This means that darcs won't
> notice that there were test failures on my commits. I haven't verified
> that this is the case on anything other than my mac.
> 
> The following patch makes py.test exit with a 1 when there are
> problems (or at least a list of stuff returned by main). This works
> for me, though I'm sure there is a better spot/way to do this.

thanks for the patch, it is conceptually applied now. Now 
currently py.test exits with 0 for successful runs, 1 for 
runs with failures and 2 if it was keyboard-interrupted,
modulo platform problems. 

cheers, 

    holger


From ianb at colorstudy.com  Sun May 15 03:17:23 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sat, 14 May 2005 20:17:23 -0500
Subject: [py-dev] Python Paste testing and py.test
Message-ID: <4286A323.1020509@colorstudy.com>

You guys might be interested in this description of using py.test and 
Paste to test applications:

   http://pythonpaste.org/docs/testing-applications.html

In the end the code behind it is fairly simple, but it seems to work 
well enough.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org


From ianb at colorstudy.com  Mon May 16 00:23:53 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 15 May 2005 17:23:53 -0500
Subject: [py-dev] Bug in assert statement
Message-ID: <4287CBF9.2030403@colorstudy.com>

When you do something like:

     assert 0, dir(module)

You get an exception from the assert reinterpretation.  Asserting with a 
non-string message normally works fine.

I've also noticed a problem when your assert statement contains an 
object who's __repr__ returns a string with a newline in it.

I've put a test in the repository at 
http://codespeak.net/svn/user/ianb/py_test_assert_failure/test_asserts.py

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org


From ianb at colorstudy.com  Mon May 16 06:28:13 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 15 May 2005 23:28:13 -0500
Subject: [py-dev] Catching KeyboardInterrupt
Message-ID: <4288215D.4080503@colorstudy.com>

It would be nice if py.test didn't so cleanly exit from a 
KeyboardInterrupt -- right now I'm encountering a test that hangs, which 
I usually figure out by ^C'ing and seeing what the traceback says.  At 
least, that's what I've become used to doing.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org


From hpk at trillke.net  Mon May 16 09:21:41 2005
From: hpk at trillke.net (holger krekel)
Date: Mon, 16 May 2005 09:21:41 +0200
Subject: [py-dev] Catching KeyboardInterrupt
In-Reply-To: <4288215D.4080503@colorstudy.com>
References: <4288215D.4080503@colorstudy.com>
Message-ID: <20050516072141.GH5609@solar.trillke.net>

On Sun, May 15, 2005 at 23:28 -0500, Ian Bicking wrote:
> It would be nice if py.test didn't so cleanly exit from a 
> KeyboardInterrupt -- right now I'm encountering a test that hangs, which 
> I usually figure out by ^C'ing and seeing what the traceback says.  At 
> least, that's what I've become used to doing.

I think the use cases are about split: sometimes you hit
KeyboardInterrupt because you realized you didn't want to run
this command and sometimes you hit it because you want to
break out of a loop. 

So now KeyboardInterrupt passes through if verbose > 0.  
Does that make sense to you? 
    
    holger


From hpk at trillke.net  Mon May 16 09:50:05 2005
From: hpk at trillke.net (holger krekel)
Date: Mon, 16 May 2005 09:50:05 +0200
Subject: [py-dev] Bug in assert statement
In-Reply-To: <4287CBF9.2030403@colorstudy.com>
References: <4287CBF9.2030403@colorstudy.com>
Message-ID: <20050516075005.GI5609@solar.trillke.net>

On Sun, May 15, 2005 at 17:23 -0500, Ian Bicking wrote:
> When you do something like:
> 
>     assert 0, dir(module)
> 
> You get an exception from the assert reinterpretation.  Asserting with a 
> non-string message normally works fine.

Sorry, i am not seeing fully what you mean: 
both tracebacks here look fine to me: 

    def test_assert_dict():
        E       assert 0, globals()
        >       AssertionError: {<dict contents left out>}

    def test_assert_nameerror():
        E       assert 0, dir(module)
        >       NameError: global name 'module' is not defined

> I've also noticed a problem when your assert statement contains an 
> object who's __repr__ returns a string with a newline in it.

That is indeed a bug. 

> I've put a test in the repository at 
> http://codespeak.net/svn/user/ianb/py_test_assert_failure/test_asserts.py

Damn, we really need a bug/issue tracker ... 

I have committed your tests modified to py/magic/test/test_assertion.py
at the end of the file.  Feel free to define what you'd like to see 
actually and commit it directly.  Next time, you can just add a test 
directly in the py lib (with an XXX prefix so it doesn't get collected. 
With proper keyword/selection support you will be able to mark 
it simply with 'pendingbug' ...) 

thanks, 

    holger


From tom at livelogix.com  Tue May 17 06:58:19 2005
From: tom at livelogix.com (Tom Locke)
Date: Tue, 17 May 2005 10:28:19 +0530
Subject: [py-dev] greenlet.main_greenlet
Message-ID: <428979EB.6000000@livelogix.com>

Hi,

I had a need to be able to test if the current greenlet was the main
greenlet. I figured this might be something that crops up now and again,
so I added a module attribute `main_greeenlet` (also copied to the type
object), so I can just test for:

    greenlet.getcurrent() == greenlet.main_greenlet

There's two small changes to greenlet.c (see diff)

Tom

Index: greenlet.c
===================================================================
--- greenlet.c  (revision 12401)
+++ greenlet.c  (working copy)
@@ -758,7 +758,7 @@
};

static char* copy_on_greentype[] = {
-       "getcurrent", "error", "GreenletExit", NULL
+       "getcurrent", "error", "GreenletExit", "main_greenlet", NULL
};

void initgreenlet(void)
@@ -787,6 +787,8 @@
        if (ts_current == NULL)
                return;

+       Py_INCREF(ts_current);
+       PyModule_AddObject(m, "main_greenlet", (PyObject*)ts_current);
        Py_INCREF(&PyGreen_Type);
        PyModule_AddObject(m, "greenlet", (PyObject*) &PyGreen_Type);
        Py_INCREF(PyExc_GreenletError);




From imagefalling at gmail.com  Wed May 18 17:52:58 2005
From: imagefalling at gmail.com (Moe A)
Date: Wed, 18 May 2005 17:52:58 +0200
Subject: [py-dev] greenlet problems!
Message-ID: <2cca93bd0505180852678fd11e@mail.gmail.com>

i ran the distutils setup scripts for both Py and the separate one for
greenlets.  i copied and pasted the following example from
codespeak.net, but it segfaults after printing "hello world" ( "zsh:
segmentation fault (core dumped)  python green.py").  i'm running
FreeBSD 5.2.1.

from py.magic import greenlet

def test1(x, y):
    z = gr2.switch(x+y)
    print z

def test2(u):
    print u
    gr1.switch(42)

gr1 = greenlet(test1)
gr2 = greenlet(test2)
gr1.switch("hello", " world")


From bob at redivi.com  Wed May 18 23:39:23 2005
From: bob at redivi.com (Bob Ippolito)
Date: Wed, 18 May 2005 17:39:23 -0400
Subject: [py-dev] greenlet problems!
In-Reply-To: <2cca93bd0505180852678fd11e@mail.gmail.com>
References: <2cca93bd0505180852678fd11e@mail.gmail.com>
Message-ID: <E35C16A4-4A2C-472A-BDC4-45E488B657F0@redivi.com>


On May 18, 2005, at 11:52 AM, Moe A wrote:

> i ran the distutils setup scripts for both Py and the separate one for
> greenlets.  i copied and pasted the following example from
> codespeak.net, but it segfaults after printing "hello world" ( "zsh:
> segmentation fault (core dumped)  python green.py").  i'm running
> FreeBSD 5.2.1.

It's important to also state which platform this is (x86, x86-64,  
etc.), greenlets use some scary assembly black magic to perform the  
stack mangling.

-bob



From imagefalling at gmail.com  Wed May 18 23:52:33 2005
From: imagefalling at gmail.com (Moe A)
Date: Wed, 18 May 2005 23:52:33 +0200
Subject: [py-dev] greenlet problems (followup)
Message-ID: <2cca93bd05051814524081558c@mail.gmail.com>

sorry, i neglected to give additional information.

im running dual x86, and here is the backtrace from the core file:

#0  0x284a35e6 in g_switchstack () from
/usr/local/lib/python2.4/site-packages/greenlet.so
#1  0x284a3831 in g_initialstub () from
/usr/local/lib/python2.4/site-packages/greenlet.so
#2  0x284a36ff in g_switch () from
/usr/local/lib/python2.4/site-packages/greenlet.so
#3  0x284a3d74 in green_switch () from
/usr/local/lib/python2.4/site-packages/greenlet.so
#4  0x080d59ed in PyCFunction_Call ()
#5  0x08059fe0 in PyObject_Call ()
#6  0x0809f7b9 in PyEval_GetFuncDesc ()
#7  0x0809d21c in PyEval_EvalFrame ()
#8  0x0809dfdb in PyEval_EvalCodeEx ()
#9  0x0809f332 in PyEval_GetFuncDesc ()
#10 0x0809f1a4 in PyEval_GetFuncDesc ()
#11 0x0809d16c in PyEval_EvalFrame ()
#12 0x0809dfdb in PyEval_EvalCodeEx ()
#13 0x0809ab02 in PyEval_EvalCode ()
#14 0x080b8d3c in PyRun_FileExFlags ()
#15 0x080b8cf7 in PyRun_FileExFlags ()
#16 0x080b8cc1 in PyRun_FileExFlags ()
#17 0x080b7f8b in PyRun_SimpleFileExFlags ()
#18 0x080b79d2 in PyRun_AnyFileExFlags ()
#19 0x08054f36 in Py_Main ()
#20 0x0805468a in main ()
#21 0x08054582 in _start ()


From arigo at tunes.org  Sun May 22 16:26:23 2005
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 22 May 2005 16:26:23 +0200
Subject: [py-dev] greenlet.main_greenlet
In-Reply-To: <428979EB.6000000@livelogix.com>
References: <428979EB.6000000@livelogix.com>
Message-ID: <20050522142623.GA20672@code1.codespeak.net>

Hi Tom,

On Tue, May 17, 2005 at 10:28:19AM +0530, Tom Locke wrote:
> I had a need to be able to test if the current greenlet was the main
> greenlet. I figured this might be something that crops up now and again,
> so I added a module attribute `main_greeenlet` (also copied to the type
> object), so I can just test for:

Unfortunately, this is not thread-safe.  A trick: the main greenlet can
be detected as the one that has no parent:

    greenlet.getcurrent().parent is None

Every thread has its own set of greenlets (including a main one).


A bientot,

Armin


From tom at livelogix.com  Mon May 23 08:03:52 2005
From: tom at livelogix.com (Tom Locke)
Date: Mon, 23 May 2005 11:33:52 +0530
Subject: [py-dev] Greenlet debugging
In-Reply-To: <20050522142623.GA20672@code1.codespeak.net>
References: <428979EB.6000000@livelogix.com>
	<20050522142623.GA20672@code1.codespeak.net>
Message-ID: <42917248.3030203@livelogix.com>

Hi,

I'm having a bash at adding debugging support for greenlets.

Specifically, I'd like to get a callback on the global trace function, 
with the `event` arg set to "switch", whenever a greenlet switch takes 
place (just after).

I'm also going to build a pdb.Pdb subclass that provides some support 
for this. (e.g. a "sw" command, meaning "run until next switch").

Any comments / guidance appreciated

 - Is this a good idea?

 - Any reason this can't work?

 - Will there be difficulties making this thread safe?

Thanks

Tom.



From ianb at colorstudy.com  Wed May 25 22:28:12 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 25 May 2005 15:28:12 -0500
Subject: [py-dev] Idea: pluggable comparison summaries
Message-ID: <4294DFDC.6090207@colorstudy.com>

It occurs to me that it would be nice to be able to plug different 
outputs/reports into the system, when encountering the idiomic form "x 
== y" (or !=).

For instance, one might use diff format when the strings above a certain 
length.  Or for large dictionaries, you might summarize differences 
(e.g. "x is missing key 'foo'", or "x['foo'] != y['foo']", etc.

I imagine these would be enabled in conftest.py somehow, and there'd be 
some list of them where each one could return a report or defer (maybe 
returning None).

I imagine there could be some nice hooks into the tk frontend at some 
point too, where you can display the data in some GUI-specific way. 
There should be room in the API for that somehow (though the tk frontend 
should degrade to showing strings for non-GUI-aware comparisons).

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org


From tom at livelogix.com  Tue May 17 06:45:26 2005
From: tom at livelogix.com (Tom Locke)
Date: Tue, 17 May 2005 04:45:26 -0000
Subject: [py-dev] greenlet.main_greenlet
Message-ID: <428976CC.2040706@livelogix.com>

Hi,

I had a need to be able to test if the current greenlet was the main 
greenlet. I figured this might be something that crops up now and again, 
so I added a module attribute `main_greeenlet` (also copied to the type 
object), so I can just test for:

    greenlet.getcurrent() == greenlet.main_greenlet

There's two small changes to greenlet.c (see diff)

Tom

Index: greenlet.c
===================================================================
--- greenlet.c  (revision 12401)
+++ greenlet.c  (working copy)
@@ -758,7 +758,7 @@
 };

 static char* copy_on_greentype[] = {
-       "getcurrent", "error", "GreenletExit", NULL
+       "getcurrent", "error", "GreenletExit", "main_greenlet", NULL
 };

 void initgreenlet(void)
@@ -787,6 +787,8 @@
        if (ts_current == NULL)
                return;

+       Py_INCREF(ts_current);
+       PyModule_AddObject(m, "main_greenlet", (PyObject*)ts_current);
        Py_INCREF(&PyGreen_Type);
        PyModule_AddObject(m, "greenlet", (PyObject*) &PyGreen_Type);
        Py_INCREF(PyExc_GreenletError);