From ianb at colorstudy.com Fri Nov 4 05:45:02 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 03 Nov 2005 22:45:02 -0600 Subject: [py-dev] py.test: initializing test environment Message-ID: <436AE74E.7060506@colorstudy.com> I want to add an option that effects the environment before any tests are collected. But I'm having a hard time doing that. I'm thinking that conftest.py files could have a setup_tests() function, that will be called with no arguments after the configuration has been parsed. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From jan at balster.info Fri Nov 4 11:14:11 2005 From: jan at balster.info (Jan Balster) Date: Fri, 04 Nov 2005 11:14:11 +0100 Subject: [py-dev] py.test: initializing test environment In-Reply-To: <436AE74E.7060506@colorstudy.com> References: <436AE74E.7060506@colorstudy.com> Message-ID: <436B3473.4030100@balster.info> Hi Ian, I'm not sure if this code snippet is the solution you wanted but I'll provide it anyway :-) Jan Ian Bicking wrote: > I want to add an option that effects the environment before any tests > are collected. But I'm having a hard time doing that. conftest.py: ------------------------------------------------------------ import py # add a new option Option = py.test.Config.Option option = py.test.Config.addoptions("my new testing options", Option('', '--setuptests', action="store_true", dest="setup_tests", default=False, help="setup tests" ) ) def setup_tests(): pass # hooking into py.test Directory collector's chain ... class SetupDirectory(py.test.collect.Directory): def __init__(self, fspath, parent=None): super(SetupDirectory, self).__init__(fspath, parent) # create files and dirs if option.setup_tests: setup_tests() Directory = SetupDirectory ---------------------------------------------------------------- If you just want to create some data files you can use setup() and teardown(): conftest.py: ----------------------------------------------------------------- ... # hooking into py.test Directory collector's chain ... class SetupDirectory(py.test.collect.Directory): mypath = py.magic.autopath().dirpath().join('my_new_file') #setup and teardown are probably invoked multiple times def setup(self): # filenames are already collected # new testfiles created here are NOT executed if option.setup_tests: #create datafiles self.mypath.ensure() def teardown(self): #teardown_tests() # remove datafiles if mypath.check(exists= True): mypath.remove() Directory = SetupDirectory --------------------------------------------------------------------------- simple test, fails if py.test is invoked whitout the new --setuptests option (test_file_was_created.py): ----------------------------------------------------------------- import py def test_file_exists(): # fail if 'my_new_file' does not exist file = py.magic.autopath().dirpath().join('my_new_file') assert file.check(exists=True) ------------------------------------------------------------------ From ianb at colorstudy.com Fri Nov 4 17:15:54 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 04 Nov 2005 10:15:54 -0600 Subject: [py-dev] py.test: initializing test environment In-Reply-To: <436B3473.4030100@balster.info> References: <436AE74E.7060506@colorstudy.com> <436B3473.4030100@balster.info> Message-ID: <436B893A.8070605@colorstudy.com> Jan Balster wrote: > Hi Ian, > > I'm not sure if this code snippet is the solution you wanted but I'll provide it > anyway :-) ... > # hooking into py.test Directory collector's chain ... > class SetupDirectory(py.test.collect.Directory): > > def __init__(self, fspath, parent=None): > super(SetupDirectory, self).__init__(fspath, parent) > # create files and dirs > if option.setup_tests: > setup_tests() > > Directory = SetupDirectory > ---------------------------------------------------------------- This is actually what I initially tried, though I only overrode .run(), not .__init__(). At least with .run() it didn't get invoked when I specified an exact file to test. Does Directory.__init__ always get invoked? -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From jan at balster.info Fri Nov 4 18:08:29 2005 From: jan at balster.info (Jan Balster) Date: Fri, 04 Nov 2005 18:08:29 +0100 Subject: [py-dev] py.test: initializing test environment In-Reply-To: <436B893A.8070605@colorstudy.com> References: <436AE74E.7060506@colorstudy.com> <436B3473.4030100@balster.info> <436B893A.8070605@colorstudy.com> Message-ID: <436B958D.3080508@balster.info> Ian Bicking wrote: > Does Directory.__init__ always get invoked? If all subdirs are valid python packages (__init__.py exists), yes. Otherwise Directory.__init__ gets invoked only if you specify a directory. If you are interested in the reason, look at py/test/collect.py:getfscollector(line 36). There is a special case: when the specified module has no package path (returned by py/path/local/local.py:pypkgpath(line 333)) the module is used without using the directory collector. Perhaps that should be changed, so the "Directory" from conftest is always used. I have to write some tests (heh, it's a testing framework ;-)) to see if this change doesn't break something else. Have a good weekend, Jan From tanner at real-time.com Fri Nov 11 19:46:17 2005 From: tanner at real-time.com (Bob Tanner) Date: Fri, 11 Nov 2005 12:46:17 -0600 Subject: [py-dev] Intent To Package (ITP) for Debian Message-ID: <200511111246.18644.tanner@real-time.com> Hello upstream developers of pylib. I'm working on a debian package for pylib and have some feedback. Please do not take any of my comments personally. I'm just trying to make my job easier :-) at the same time make recommendations on how to make pylib more Debian compliant. 1. svn snapshot from 2005/11/11 has .gif/.png in /usr/lib, according to Filesystem Hierarchy Standard (FHS) architecture-independent files need to be placed within /usr/share instead 2. Finding python scripts that by default aren't executable on installation, should these scripts be executable? /usr/lib/python2.4/site-packages/py/compat/testing/_findpy.py /usr/lib/python2.4/site-packages/py/compat/testing/test_optparse.py /usr/lib/python2.4/site-packages/py/misc/cmdline/countloc.py /usr/lib/python2.4/site-packages/py/compat/testing/_findpy.py and /usr/bin/_findpy.py (see below) look to be the same script, can one or the other be removed? 3. The install puts /usr/bin/pytest.cmd in place on linux, this looks to be a DOS batch file, I don't think it should be installed when the os is linux. 4. Debian requires anything in /usr/bin to have a manpage. I have found help2man to be very useful and maintaining manpages from the -h (--help) and -v (--version) of the program. /usr/bin/_findpy.py /usr/bin/py.cleanup /usr/bin/py.lookup /usr/bin/py.test /usr/bin/_makepyrelease.py /usr/bin/py.countloc /usr/bin/py.rest I'd also like feedback on the _findpy.py and _makepyrelease.py, should they be in a "production" package or are the development tools? The underscore breaks Debian executable naming policy. 5. Good news is even with the Debian "problems" the code works great. $ py.test test/ tests finished: 138 passed in 19.18 seconds I'm happy to submit patches if you are willing to accept them for review. I much rather have things put into upstream code then maintain separate patches for Debian. Look forward to your (collective) responses. Thanks. -- Bob Tanner | Phone : (952)943-8700 http://www.real-time.com, Minnesota, Linux | Fax : (952)943-8500 Key fingerprint = AB15 0BDF BCDE 4369 5B42 1973 7CF1 A709 2CC1 B288 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From pinard at iro.umontreal.ca Sun Nov 13 05:36:18 2005 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Sat, 12 Nov 2005 23:36:18 -0500 Subject: [py-dev] py.log suggestion In-Reply-To: <20051028083041.GI25572@hpk.local> References: <20051015004603.GA10584@alcyon.progiciels-bpi.ca> <20051015082141.GK23762@solar.trillke.net> <20051015125830.GA24173@alcyon.progiciels-bpi.ca> <20051015131107.GM23762@solar.trillke.net> <20051016163704.GA12038@phenix.progiciels-bpi.ca> <20051022191809.GE500@hpk.local> <20051026022223.GA9498@alcyon.progiciels-bpi.ca> <20051028083041.GI25572@hpk.local> Message-ID: <20051113043618.GA24282@phenix.sram.qc.ca> [holger krekel] >Hey Francois, Hi, Holger! Are you back, or still enjoying the travel? :-) >> > So, in short, i propose to treat the case of the first arg being >> > a callable, simply as: >> > if consumer is not None: >> > if args and callable(args[0]): >> > args = args[0](*args[1:], **kwargs) >> > kwargs = None >> > ... >> It surely solves the initial problem, anyway, which is to have some way >> to conditionalise out semi-lengthy computations whose sole purpose is to >> generate the text of log messages. Well, in the big program we've started cleaning a few months ago, and which I'll surely continue cleaning for months, the above would take care of most situations I saw so far. Yet, I found a case where it would not be sufficient, because the fact that logging will occur or not changes the algorithm long in advance. That case is sketched at the end of this message, in case it helps figuring out what I mean :-). However, you know, this is only a seek for elegance. The program already accepts a flurry of options for controlling lists and traces of all kinds, and I'm trying to get rid of many of these options by sticking to the py.log paradigm instead. For a few unusual cases, I could either keep some of these options, or add interceptions in the place where py.log consumers are decided for setting a few flags to be tested later. There is no real problem, and I'm a reasonable man! Yet, the extra flags are conceptually unneeded, if py.log offered an API for testing a particular trace is active or not. If easily doable, the nicest that come to mind is still using the logger in boolean context. The idea of the first argument being callable is attractive indeed, and I really find the approach much nicer than salting the source with a lot of extraneous tests. Yet, there are a few cases where I think it would be nice to test, as elegantly as possible, if a logger has a consumer. >> And there is this unexpected danger as well, which you underlined in one >> of your replies, and which I did not foresee (I should probably >> have), that mylog may receive a first argument which is unexpectedly >> callable. >Right. So i have a new suggestion. > log.info[arg1, arg2, ...] >would behave exactly like the log.info(arg1, arg2, ...) does now, >i.e. a dumb "print" emulation. No special check for callables, >formatting and such. Not bad! I like this! :-) >But then we can put the "call function" syntax to new usage. So with > log.info(*args, **kwargs) >we would now have special treatment (rough sketch): > ... > if consumer is not None: > if callable(args[0]): > consumer(args[0](*args[1:], **kwargs)) > else: > if kwargs: > assert len(args) == 1, "..." > consumer(args[0] % kwargs) > else: > assert not kwargs, "..." > consumer(args[0] % args[1:]) >this should IMO allow a rather intuitive usage for percent-formatting >as well as for defering formatting to custom callables. Agreed, and quite interesting. >This is unfortunately a somewhat incompatible change to current usage >but well, it's not used that much yet and we probably have enough >access to the code bases that currently use it. It surely would not be a practical problem for me / us, in any case. Here is, for the curious, a short description of the case where I would need to test if a particular trace is active or sent to /dev/null, and where giving a callable to a py.log logger would not be a solution. During its initialisation, the application builds an internal tree from a parameter file using a compact format. This file, without being huge, is not small. Later in the application, this tree (or parts thereof) is repeatedly "evaluated", given varying evaluation contexts. In a typical run, such evaluations occur a million times or so, and for some of these evaluations, hundreds of nodes need to be visited. So, special attention is given for the evaluation to be done efficiently, and the digested tree gets "optimized" in many ways before being used. Jacques, who writes the parameter file (that one digested into a tree) sometimes needs to understand why or how a particular decision is reached by the evaluator (routinely called "the engine", here). So he asks the operator to launch the application over a fairly limited subset of the whole input and to activate the "explanation mode" of the engine. In this mode, whenever a tree evaluation occurs, a copy of the big tree is recreated, yet massively pruned from all subtrees which did not participate in that particular evaluation, in the given context. That subtree is then printed for later study and analysis, together with related trace information. The trace itself contains textual reference locators, already built into the tree by the parser/scanner pair digesting the parameter file. The key here is that in "explanation mode", a lot of shortcuts and optimisations are inhibited, so the trace gets more decipherable, and the re-built trees cleaner. Performance considerations forces the application to use a wholly rewritten evaluator in "explanation mode", so the normal evaluator is not slowed down by tests, traces, and sub-tree rebuilding. The selection of the proper evaluator depends on if "explanation mode" is wanted or not. Despite a lot of mechanics is involved for this, I would like to see it, at least operationally, as nothing more than just another mere trace, among many other mere traces. -- Fran?ois Pinard http://pinard.progiciels-bpi.ca From cfbolz at gmx.de Tue Nov 15 23:19:51 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 15 Nov 2005 23:19:51 +0100 Subject: [py-dev] failing import in toplevel module Message-ID: <437A5F07.9060809@gmx.de> Hi py-dev! today I had the following problem with py.test: I could not find a way to skip tests in a package if the import of the package itself fails (because the __init__.py raises something). Example: assuming I have a package directory "example" with two files in it: __init__.py: ------------ something that can raise ImportError test_it.py: ----------- import py try: import example except ImportError: py.test.skip("missing dependencies, whatever") ... (some tests) In this situation the tests will always fail because py.test automatically tries to import the top-level package. This cannot be circumvented even when using a conftest file since the import of the top-level package happens very early. Is there something I was missing? Couldn't for example py.test only insert the path to the top-level dir into sys.path, so that the "import example" find the correct package? Cheers, Carl Friedrich From ianb at colorstudy.com Mon Nov 21 21:16:53 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 21 Nov 2005 14:16:53 -0600 Subject: [py-dev] Moving py-lib to setuptools Message-ID: <43822B35.5080904@colorstudy.com> Does anyone have an opinion on setup.py and how py-lib is packaged? It seems like it was put in place some time ago, and py hasn't really been formally released since then. Anyway, I think a lot of the hackish parts are dealt with by setuptools. setuptools.find_packages() fills out the packages, and its backport of package_data deals with non-Python files, and paste.util.finddata can be copied in to deal with enumerating the package data. setuptools has some possibly useful script generation capability, but that doesn't need to be used. Maybe it should be, though; it creates a .exe file on Windows for each script. I don't know know it would interact with the "py.test" name; I doubt that's been considered on Windows, and probably setuptools should remove dots from scripts on Windows. I was going to upload the setup.py file to use setuptools, but I lost my svn password, so I'll just attach them for now. There's probably some other things I'd add to fully convert it, like just removing the download URL and using the Cheese Shop for file distribution, and perhaps adding a setup.cfg that append svn revisions to the versions of distributions made from the trunk. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: finddata.py URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: setup.py URL: From pobrien at orbtech.com Mon Nov 21 22:01:02 2005 From: pobrien at orbtech.com (Patrick K. O'Brien) Date: Mon, 21 Nov 2005 15:01:02 -0600 Subject: [py-dev] Moving py-lib to setuptools In-Reply-To: <43822B35.5080904@colorstudy.com> References: <43822B35.5080904@colorstudy.com> Message-ID: <4382358E.6010102@orbtech.com> Ian Bicking wrote: > Does anyone have an opinion on setup.py and how py-lib is packaged? It > seems like it was put in place some time ago, and py hasn't really been > formally released since then. > > Anyway, I think a lot of the hackish parts are dealt with by setuptools. > setuptools.find_packages() fills out the packages, and its backport of > package_data deals with non-Python files, and paste.util.finddata can be > copied in to deal with enumerating the package data. setuptools has > some possibly useful script generation capability, but that doesn't need > to be used. Maybe it should be, though; it creates a .exe file on > Windows for each script. I don't know know it would interact with the > "py.test" name; I doubt that's been considered on Windows, and probably > setuptools should remove dots from scripts on Windows. I just renamed a setuptools-generated script and exe to contain a dot in the name and it worked fine on Windows XP. Just FYI. -- Patrick K. O'Brien Orbtech http://www.orbtech.com Schevo http://www.schevo.org From rudazz at gmail.com Wed Nov 30 22:42:47 2005 From: rudazz at gmail.com (Ruda Filgueiras) Date: Wed, 30 Nov 2005 19:42:47 -0200 Subject: [py-dev] Call py.test.cmdline.main() from a python script Message-ID: <776143310511301342q665e47d7h@mail.gmail.com> Hello, I'm using py.test to run my unit tests for STOQ 2 [1] project. To run my tests I can execute py.test script, but I nedd to call py.test from a python script that initalize database and executes some doctests using doctest module. To make this, I add this simple code inside my script: +import py +py.test.cmdline.main() My enviroment uses simbolic links from package/module to a directory on my PYTHONPATH. And after using my runtests.py script with these lines, py.test dont import my test modules correctly. (see traceback log attached) Anohter member of STOQ2 staff, Johan Dahlin , verify the problem and propose a one line patch to this problem, and I'm send this patch attached too. Thanks, 1. STOQ2 is a Open Source Reatail System written in python and uses: - pygtk - kiwi (MVC framework on top of pygtk) - SQLObject - py.test (pylib) - twisted component model -- Rud? Porto Filgueiras -------------- next part -------------- A non-text attachment was scrubbed... Name: py_test.patch Type: text/x-patch Size: 562 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: err.log Type: text/x-log Size: 29910 bytes Desc: not available URL: