From holger at merlinux.eu Tue Nov 3 21:52:41 2009 From: holger at merlinux.eu (holger krekel) Date: Tue, 3 Nov 2009 21:52:41 +0100 Subject: [py-dev] execnet-1.0.0b1 released: now with Python3/Jython support Message-ID: <20091103205241.GO29141@trillke.net> Hi everybody! i just uploaded a first public release of execnet, namely 1.0.0b1 to PyPI. Grab it with easy_install/pip and your favourite Python environment (no setuptools or distribute required). execnet allows to ad-hoc instantiate local and remote Python interpreters, currently Python2.4, 2.5, 2.6 and (new) also Python 3.1, Jython and PyPy-c. It works on Windows, Linux and OSX and is licensed under the GPL V2 or later. With execnet you can instantiate "gateways" between Python processes, use "remote_exec" code execution and "channels" to perform structured data communication. See here for more info: http://codespeak.net/execnet Here is a list of changes from the rather internal 1.0.0alpha2 release: * added new examples for NumPy, Jython, IronPython * improved documentation * include apipkg.py for lazy-importing * integrated new serializer code from Benjamin Peterson * improved support for Jython-2.5.1 have fun and let me know if you have issues, holger -- Metaprogramming, Python, Testing: http://tetamap.wordpress.com Python, PyPy, pytest contracting: http://merlinux.eu From holger at merlinux.eu Wed Nov 4 11:34:21 2009 From: holger at merlinux.eu (holger krekel) Date: Wed, 4 Nov 2009 11:34:21 +0100 Subject: [py-dev] Log to a file In-Reply-To: <18738-78272@sneakemail.com> References: <21883-03418@sneakemail.com> <20091028084400.GG29141@trillke.net> <18738-78272@sneakemail.com> Message-ID: <20091104103421.GR29141@trillke.net> Hi Simon, i finally got around to look at your plugin ... On Thu, Oct 29, 2009 at 17:42 +1100, Simon wrote: > On 28/10/2009, at 19:44 , holger krekel holger-at-merlinux.eu |py-dev + > execnet-dev| wrote: > >>> Hi Holger, >>> >>> I started looking into this a bit but ran into a problem when I >>> started to probe the py.io module. >>> >>> I tried running the py.io examples on the website >>> but when I run the py.io.StdCaptureFD example, python dies on the >>> second line: >>> >>> stakita at okum:~$ python >>> Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) >>> [GCC 4.3.3] on linux2 >>> Type "help", "copyright", "credits" or "license" for more >>> information. >>> History restored: /auto/home/stakita/.pyhistory, Max Length: 500 >>>>>> import py, sys >>>>>> capture = py.io.StdCaptureFD() >> >> The redirect here happens for stdout, stderr and stdin is put to >> /dev/zero on the file descriptor level ... >> >>> stakita at okum:~$ >> >> ... so the python interpreter immediately gets an EOL on stdin and >> dies. > > So does this mean that the example is a bit broken then? yes, i fixed it in trunk. >>> My guess is that an exception is being thrown, but the captured >>> sys.stderr descriptor is not passing any meaningful traceback. >> >> you can call py.io.StdCaptureFD(err=True, in_=False, >> out=False) and should be able to play with writing to stderr. >> >>> I tried wrapping the call in try/except, but that didn't seem to >>> help. >>> On the positive side, the problem appears to be quite reproducible. >>> >>> Any ideas on what is going on here? >> >> just anoter recommendation. Maybe it is easiest for you to implement >> a "--iocapturelog=filename" option and see to send output to >> it from the pytest_capture plugin on the py-trunk repository. >> You can start experimenting by copying the pytest_capture.py >> file somewhere into your import path and modifying it. Your >> local version will override the builtin one. >> >> HTH, >> holger > > Thanks for the suggestion, and here is a first pass. Attached is a > modified version of pytest_capture.py from the 1.0.2 release. > > In this file, I just push the options into the CaptureManager so that > the command line args are available. This does mean that the config file > options stuff is not hooked up, but at least it works from the command > line at the moment. just using "config.getvalue()" should be fine. "py.test --help-config" then shows the derived ENV/conftest.py settings, btw. > Additional plugin command line options are: > --log-file > When used with the two capture options ('fd' and 'sys') this > will enable pushing the stdout and stderr writes to the specified file. > --append-log > This allows appending to the file as opposed to the default > clobber behaviour. > > Does this fit with your model of how the plugin should be structured? roughly, yes :) some small comments: * the "append-log" option i'd rather encode as "+logfilename" for the capture-file option (goal is to minimize number of cmdline args) just one --capture-file option makes sense * "self.logfd" is actually a file object, not a file descriptor, so better use "self.capture_file" or so. * your code is not Python3 compatible I'd like to include your changes ... could you rework your patch by checking out http://bitbucket.org/hpk42/py-trunk and modifying _py/test/plugin/pytest_capture.py and running and adding tests here: testing/pytest/plugin/test_pytest_capture.py ? FYI on linux/OSX you can do alias py.test='python /path/to/checkout/bin/py.test' alias py.test3='python3 /path/to/checkout/bin/py.test' to easily have the two main py.test runs available ... adding a py.testjython alias is a bonus :) In any case, feel free to ask more questions, also on #pylib freenode. cheers, holger From holger at merlinux.eu Wed Nov 4 18:23:42 2009 From: holger at merlinux.eu (holger krekel) Date: Wed, 4 Nov 2009 18:23:42 +0100 Subject: [py-dev] py.test : setup / teardown at the directory level In-Reply-To: <4AE80A42.5030009@freehackers.org> References: <49F6EA81.2070106@freehackers.org> <20090428115556.GJ11963@trillke.net> <49F71C0E.3000800@freehackers.org> <20090428180200.GO11963@trillke.net> <4AE80A42.5030009@freehackers.org> Message-ID: <20091104172342.GU29141@trillke.net> Hi Philippe, On Wed, Oct 28, 2009 at 10:09 +0100, Philippe Fremy wrote: > holger krekel wrote: > > Hi Philippe, > > > > your described use case makes lots of sense to me. > > > > I coded an example which i hope fits it. > > > > It uses the new "local plugins" (i.e. plugins defined in a > > conftest.py) and funcargs, if you don't know about them > > yet i hope this is good to skim/read first: > > http://codespeak.net/py/trunk/test/funcargs.html > > > > Here is the example: > > > > http://bitbucket.org/hpk42/py-trunk/src/tip/example/funcarg/lazysetup/ > > > > using py-trunk (probably also works with the 1.0.0b1, haven't checked) > > in the lazysetup directory you can now do > > > > py.test sub1 # will wait 5 seconds because test > > # functions access the setup defined in > > # conftest.py > > > > py.test sub2 # will immediately run as the "setup" > > # funcarg is not requested > > > > The idea for this conftest.py implementation is simple: > > setup the funcarg when first needed and only tear it down > > when the test process exits. > > > > does this make sense to you? feel free to play around > > and ask questions - I'd then put the above example into > > the "tutorial" example section of the funcarg doc. > > > > One advantage of the above approach is that you do not > > need to do anything in your test modules anymore > > (no boilerplate importing of setup_module etc.) > > than requesting the object you want to setup. > > > I am reviving this old thread. > > Honestly, I haven't tried the funcargs based solution that you propose. > The reasons are : > - I would really prefer to have setup/teardown at directory level and > your solution is more per-session level true, that's currently the case. > - I don't like the idea of modifying 100 tests just to get a > setup/teardown effect Sure. i am thinking about introducing a general "pytest_pyfunc_setup" hook that you could define at project, directory, module or class level and that would be called for setup of each function. > - I still find funcargs a bit cumbersome as explained in my other mail. > > I had a quick look at the plugin architecture to see if I could > implement an equivalent of setup/teardown at directory level, but I > don't think it's possible. Whatever is called for "directory" setup could live in a conftest.py file. The question is how to transfer any "directory" setup state to a module. At first i thought one could write down: # ./conftest.py def setup_module(mod): # will be called for all modules in/below the dir mod.something = "value" but one would expect this to be called for each module and not just once for a whole directory. Now one could perform some directory-level caching but one conceptual issue remains: values would be "magically" injected into the test modules. Do you have ideas about how you'd like the API to work? > Can you consider this as a feature request ? i'd like to get an idea on how this could work at all conceptually. best, holger From holger at merlinux.eu Wed Nov 4 18:57:18 2009 From: holger at merlinux.eu (holger krekel) Date: Wed, 4 Nov 2009 18:57:18 +0100 Subject: [py-dev] using funcargs for setup/teardown In-Reply-To: <4AE7EE2E.1020208@freehackers.org> References: <4ADE0957.7010005@gmx.net> <20091020210933.GG29141@trillke.net> <4AE459EC.805@gmx.net> <20091026102628.GB29141@trillke.net> <4AE7EE2E.1020208@freehackers.org> Message-ID: <20091104175718.GV29141@trillke.net> Hi Philippe, On Wed, Oct 28, 2009 at 08:09 +0100, Philippe Fremy wrote: > holger krekel wrote: > > Hi Frederik! > > > >> However, I've finally figured out why I'm having such a hard time > >> warming up to funcargs: They go way beyond the minimalist simplicity > >> that made me switch to py.test in the first place. > >> (I realize there's some Magic to py.test's internals, but that doesn't > >> surface in the API.) > >> > >> Funcargs seem like a departure from this principle. > >> I understand that some situations demand such complexity, so I'm not > >> arguing against funcargs in general - but for my part, I've managed to > >> keep it simple so far. > >> > I second Frederic here. I chose py.test because of the overall > simplicity of the framework when you use it. The > setup_module/setup_class/setup_method without any special magic other > than using function, class or method names starting with test is really > really nice. I am kind of worried because I could no longer find them in > the documentation. I eventually found them in the xUnit documentation > but that's not where I expected them. I vote for a return of this > documentation to the main py.test documentation, and moving funcarg > related documentation to an "advanced test setup" section. I am worried that newcomers will be confused then. But i see the point of "unhiding" the xUnit documentation. > funcarg seems to be a powerful tool, but really cumbersome to grok. I > find the magic trick on the naming with __myargument a bit cumbersome. I > would feel more comfortable with a syntax that mimic the setup/teardown > used for module, class and methods. It would be easy to also allow explicit registration for factories ala: def pytest_configure(config): config.add_funcarg_factory(name, factoryfunc) but at the time i choose "convention over configuration" and the "__" because Django also uses a similar way to encode values into a function name. > > This implies having to call magic methods for setting up objects in > > global namespaces or 'self' attributes - which not only makes the test > > harder to understand and refactor IMO. > > I do not agree here. If I have one class with 30 unit test methods, it's > easier to setup/teardown the test parameters in two methods for the > whole class than modifying 30 test methods to add funcargs arguments. agreed, i am considering introducing a new hook: def pytest_pyfunc_setup(request): request.cached_setup(setupfunc, teardownfunc, scope="directory") and this hook would be called for each python test function. Here we call the cached_setup helper to help us manage setup/teardown scopes. (see http://tinyurl.com/yfw82l5 for request object attributes which you could pass into your setup/teardown func) It would also mean you could write down: class TestGroup: parameter = 3 def test_method(self): assert self.parametrized_obj ... and implement a pytest_pyfunc_setup() to set a "self.someobj" according to the class-specified parameter. Would this make sense to you? > So, while I agree that funcargs certainly has potential, I think you > should not force it onto the user and should really stress the two ways > of setting up per class or per method parameters. ok, I'll see to work on the docs a bit. > And it would be really nice to figure out a syntax for funcarg that is > more in the setup/teardown fashion. > > I don't see much gain of using : > > def pytest_funcarg__mysetup(request): > return MySetup(request) > > class TestClass: > def test_function(self, mysetup): > conn = mysetup.getsshconnection() > # work with conn > > > instead of : > > class TestClass: > def setup_class( c ): > c.mysetup = MySetup() > > def test_function(self): > conn = c.mysetup.getsshconnection() > # work with conn > > > The programming style is different, the second one is traditional OO and > will be familiar to anybody coming from C++, Java or other OO world. The > second one is playing more on the python capabilities. I don't find it > more readable. If you have setup_module + setup_class + setup_method (+ subclassing!) layers of setup and teardown, things get a bit harder to understand. In >3000 test projects i know off this led to complexity and refactoring difficulties. > One can argue about the advantage that each test function can take a > different parameter. While true, in my testing experience, I haven't > seen a pattern with many different tests taking many different > parameters. I usually have groups of 5 to 10 tests taking one kind of > parameter. If they need to take another kind of parameter, I will put > them in a different test class. These days i sometimes group tests by feature rather than by setup-parameters. E.g. i put two unit-tests and one functional test into a "TestFeature" class. > So, in my opinion, using funcargs for regular cases is a matter of style. > > I do see a big value in funcargs for the parameterized tests. Running > the same test over and over with different parameter is a really nice > feature. I think you should stress it more in the documentation. makes sense. thanks a lot for your feedback! best, holger From holger at merlinux.eu Thu Nov 5 03:34:26 2009 From: holger at merlinux.eu (holger krekel) Date: Thu, 5 Nov 2009 03:34:26 +0100 Subject: [py-dev] py-test-1.1.0 candidate Message-ID: <20091105023426.GY29141@trillke.net> Hi folks, i have a candidate tar ball for the 1.1.0 release which i'd like to release final thursday or friday. Could someone easy_install/pip http://merlinux.eu/~hpk/py-1.1.0.tar.gz with a Python2 or Python3 environment and tell me if "py.test -h" and "py.test -v" work? And maybe run it on your test suite that used to work with 1.0.2? Preliminary release announcement and docs are here, btw: http://codespeak.net/py/trunk/announce/release-1.1.0.html best, holger -- Metaprogramming, Python, Testing: http://tetamap.wordpress.com Python, PyPy, pytest contracting: http://merlinux.eu From schmir at gmail.com Thu Nov 5 11:10:49 2009 From: schmir at gmail.com (schmir at gmail.com) Date: Thu, 05 Nov 2009 11:10:49 +0100 Subject: [py-dev] py-test-1.1.0 candidate In-Reply-To: <20091105023426.GY29141@trillke.net> (holger krekel's message of "Thu, 5 Nov 2009 03:34:26 +0100") References: <20091105023426.GY29141@trillke.net> Message-ID: <871vkd46za.fsf@muni.brainbot.com> holger krekel writes: > Hi folks, > > i have a candidate tar ball for the 1.1.0 release > which i'd like to release final thursday or friday. > Could someone > > easy_install/pip http://merlinux.eu/~hpk/py-1.1.0.tar.gz > > with a Python2 or Python3 environment and tell me if "py.test -h" > and "py.test -v" work? And maybe run it on your test suite > that used to work with 1.0.2? > This all seems to work. But did you really expect "py.test -v" to break? I've tested with a 64 bit linux running python 2.6.4 and a 32 bit linux running python 2.4.6. I've also run the mwlib test suite and didn't find any issues. ,---- | [py24-test-mwlib] ~/t/ % py.test -v | ================================================================ test session starts ================================================================= | python: platform linux2 -- Python 2.4.6 -- pytest-1.1.0 -- /home/ralf/py24-test-mwlib/bin/python | test object 1: /home/ralf/t | | ================================================================== in 0.00 seconds ================================================================== | [py24-test-mwlib] ~/t/ % py.test -h | usage: py.test [options] [file_or_dir] [file_or_dir] [...] | | options: | -h, --help show this help message and exit | | running and selection options: | -x, --exitfirst exit instantly on first error or failed test. | -k KEYWORD only run test items matching the given space separated | keywords. precede a keyword with '-' to negate. | Terminate the expression with ':' to treat a match as a | signal to run all subsequent tests. | -p PLUGINS load the specified plugin after command line parsing. | --boxed box each test run in a separate process | --capture=method set capturing method during tests: fd (default)|sys|no. | -s shortcut for --capture=no. | --pdb start pdb (the Python debugger) on errors. | --pastebin=mode send failed|all info to Pocoo pastebin service. | | terminal reporting: | -v, --verbose increase verbosity. | -l, --showlocals show locals in tracebacks (disabled by default). | --report=opts comma separated reporting options | --tb=style traceback verboseness (long/short/no). | --fulltrace don't cut any tracebacks (default is to cut). | | test process debugging and configuration: | --basetemp=dir base temporary directory for this test run. | --collectonly only collect tests, don't execute them. | --traceconfig trace considerations of conftest.py files. | --nomagic don't reinterpret asserts, no traceback cutting. | --debug generate and show internal debugging information. | --help-config show available conftest.py and ENV-variable names. | --version display py lib version and import information. | --no-assert disable python assert expression reinterpretation. `---- Regards, - Ralf From holger at merlinux.eu Thu Nov 5 13:07:56 2009 From: holger at merlinux.eu (holger krekel) Date: Thu, 5 Nov 2009 13:07:56 +0100 Subject: [py-dev] py-test-1.1.0 candidate In-Reply-To: <871vkd46za.fsf@muni.brainbot.com> References: <20091105023426.GY29141@trillke.net> <871vkd46za.fsf@muni.brainbot.com> Message-ID: <20091105120756.GZ29141@trillke.net> On Thu, Nov 05, 2009 at 11:10 +0100, schmir at gmail.com wrote: > holger krekel writes: > > > Hi folks, > > > > i have a candidate tar ball for the 1.1.0 release > > which i'd like to release final thursday or friday. > > Could someone > > > > easy_install/pip http://merlinux.eu/~hpk/py-1.1.0.tar.gz > > > > with a Python2 or Python3 environment and tell me if "py.test -h" > > and "py.test -v" work? And maybe run it on your test suite > > that used to work with 1.0.2? > > > > This all seems to work. But did you really expect "py.test -v" to break? well, i am most concerned about packaging issues. And actually meant to ask for "--version" even only :) I've tested myself on XP, OSX and Linux with various Python interpreters, both py.test own tests and 3rd party ones. > I've tested with a 64 bit linux running python 2.6.4 and a 32 bit linux > running python 2.4.6. I've also run the mwlib test suite and didn't find > any issues. great, thanks for the feedback. best, holger > ,---- > | [py24-test-mwlib] ~/t/ % py.test -v > | ================================================================ test session starts ================================================================= > | python: platform linux2 -- Python 2.4.6 -- pytest-1.1.0 -- /home/ralf/py24-test-mwlib/bin/python > | test object 1: /home/ralf/t > | > | ================================================================== in 0.00 seconds ================================================================== > | [py24-test-mwlib] ~/t/ % py.test -h > | usage: py.test [options] [file_or_dir] [file_or_dir] [...] > | > | options: > | -h, --help show this help message and exit > | > | running and selection options: > | -x, --exitfirst exit instantly on first error or failed test. > | -k KEYWORD only run test items matching the given space separated > | keywords. precede a keyword with '-' to negate. > | Terminate the expression with ':' to treat a match as a > | signal to run all subsequent tests. > | -p PLUGINS load the specified plugin after command line parsing. > | --boxed box each test run in a separate process > | --capture=method set capturing method during tests: fd (default)|sys|no. > | -s shortcut for --capture=no. > | --pdb start pdb (the Python debugger) on errors. > | --pastebin=mode send failed|all info to Pocoo pastebin service. > | > | terminal reporting: > | -v, --verbose increase verbosity. > | -l, --showlocals show locals in tracebacks (disabled by default). > | --report=opts comma separated reporting options > | --tb=style traceback verboseness (long/short/no). > | --fulltrace don't cut any tracebacks (default is to cut). > | > | test process debugging and configuration: > | --basetemp=dir base temporary directory for this test run. > | --collectonly only collect tests, don't execute them. > | --traceconfig trace considerations of conftest.py files. > | --nomagic don't reinterpret asserts, no traceback cutting. > | --debug generate and show internal debugging information. > | --help-config show available conftest.py and ENV-variable names. > | --version display py lib version and import information. > | --no-assert disable python assert expression reinterpretation. > `---- > > > Regards, > - Ralf > -- Metaprogramming, Python, Testing: http://tetamap.wordpress.com Python, PyPy, pytest contracting: http://merlinux.eu From holger at merlinux.eu Thu Nov 5 18:00:43 2009 From: holger at merlinux.eu (holger krekel) Date: Thu, 5 Nov 2009 18:00:43 +0100 Subject: [py-dev] py.test/pylib 1.1.0 final released Message-ID: <20091105170043.GB29141@trillke.net> Hi, seems like all works fine except for a little pip-install issue so i just updated some docs and loaded a py-1.1.0 final to PyPi. The online release-announcement is here: http://tinyurl.com/yg5ndn3 thanks to all who helped in the process! best, holger py.test/pylib 1.1.0: Python3, Jython, advanced skipping, cleanups ... -------------------------------------------------------------------------------- Features: * compatible to Python3 (single py2/py3 source), `easy to install`_ * generalized marking_: mark tests one a whole-class or whole-module basis * conditional skipping_: skip/xfail based on platform/dependencies Fixes: * code reduction and "de-magification" (e.g. 23 KLoc -> 11 KLOC) * distribute testing requires the now separately released execnet_ package * funcarg-setup/caching, "same-name" test modules now cause an exlicit error * de-cluttered reporting options, --report for skipped/xfail details Compatibilities 1.1.0 should allow running test code that already worked well with 1.0.2 plus some more due to improved unittest/nose compatibility. More information: http://pytest.org thanks and have fun, holger (http://twitter.com/hpk42) .. _execnet: http://codespeak.net/execnet .. _`easy to install`: ../install.html .. _marking: ../test/plugin/mark.html .. _skipping: ../test/plugin/skipping.html Changelog 1.0.2 -> 1.1.0 ----------------------------------------------------------------------- * remove py.rest tool and internal namespace - it was never really advertised and can still be used with the old release if needed. If there is interest it could be revived into its own tool i guess. * fix issue48 and issue59: raise an Error if the module from an imported test file does not seem to come from the filepath - avoids "same-name" confusion that has been reported repeatedly * merged Ronny's nose-compatibility hacks: now nose-style setup_module() and setup() functions are supported * introduce generalized py.test.mark function marking * reshuffle / refine command line grouping * deprecate parser.addgroup in favour of getgroup which creates option group * add --report command line option that allows to control showing of skipped/xfailed sections * generalized skipping: a new way to mark python functions with skipif or xfail at function, class and modules level based on platform or sys-module attributes. * extend py.test.mark decorator to allow for positional args * introduce and test "py.cleanup -d" to remove empty directories * fix issue #59 - robustify unittest test collection * make bpython/help interaction work by adding an __all__ attribute to ApiModule, cleanup initpkg * use MIT license for pylib, add some contributors * remove py.execnet code and substitute all usages with 'execnet' proper * fix issue50 - cached_setup now caches more to expectations for test functions with multiple arguments. * merge Jarko's fixes, issue #45 and #46 * add the ability to specify a path for py.lookup to search in * fix a funcarg cached_setup bug probably only occuring in distributed testing and "module" scope with teardown. * many fixes and changes for making the code base python3 compatible, many thanks to Benjamin Peterson for helping with this. * consolidate builtins implementation to be compatible with >=2.3, add helpers to ease keeping 2 and 3k compatible code * deprecate py.compat.doctest|subprocess|textwrap|optparse * deprecate py.magic.autopath, remove py/magic directory * move pytest assertion handling to py/code and a pytest_assertion plugin, add "--no-assert" option, deprecate py.magic namespaces in favour of (less) py.code ones. * consolidate and cleanup py/code classes and files * cleanup py/misc, move tests to bin-for-dist * introduce delattr/delitem/delenv methods to py.test's monkeypatch funcarg * consolidate py.log implementation, remove old approach. * introduce py.io.TextIO and py.io.BytesIO for distinguishing between text/unicode and byte-streams (uses underlying standard lib io.* if available) * make py.unittest_convert helper script available which converts "unittest.py" style files into the simpler assert/direct-test-classes py.test/nosetests style. The script was written by Laura Creighton. * simplified internal localpath implementation -- Metaprogramming, Python, Testing: http://tetamap.wordpress.com Python, PyPy, pytest contracting: http://merlinux.eu From py-dev at tolomea.com Fri Nov 6 07:11:40 2009 From: py-dev at tolomea.com (Gordon Wrigley) Date: Fri, 6 Nov 2009 17:11:40 +1100 Subject: [py-dev] unclean shutdown of py.test Message-ID: Short form: I have a test that in some circumstances can't raise an exception, all it can do is call stuff. Currently it calls sys.exit, which is less than optimal. Is there a function or series of functions I can call to somewhat cleanly shutdown py.test in this situation. Longer form: First It is worth noting that we always run py.test with -x, I suspect that will have some bearing on what follows.. The application I'm testing calls a bunch of C code via ctypes. In the event of certain pathological failures that C code does an internal error operation equivalent to assert, i.e. print some stuff and then call exit. When this happens there is no way to unroll the C stack to get back to the calling python context and notify py.test of the failure. What I have managed to do is intercept the error, and from the intercept use ctypes callback functionality to call into a python handler. It would be nice if I could raise an exception there and use that to get back to the original python context, but ctypes doesn't support that, instead it captures, prints and discards the exception before returning to the C context. I have sent them an idea I have for how the exception could instead optionally be propagated past the C and back to python using setjmp / longjmp, but I'm not really hopeful that that will be a viable alternative. So what I would like to do is clean up py.test in the callback. My ideal outcome would be if I could do some small amount of stuff that would make py.test act as if it were exiting due to a failed test while running with -x. Is this line of approach at all possible? or does it violate the py.test structure too much to be workable? Gordon From holger at merlinux.eu Fri Nov 6 11:53:15 2009 From: holger at merlinux.eu (holger krekel) Date: Fri, 6 Nov 2009 11:53:15 +0100 Subject: [py-dev] unclean shutdown of py.test In-Reply-To: References: Message-ID: <20091106105315.GF29141@trillke.net> Hi Gordon, On Fri, Nov 06, 2009 at 17:11 +1100, Gordon Wrigley wrote: > Short form: > I have a test that in some circumstances can't raise an exception, all > it can do is call stuff. > > Currently it calls sys.exit, which is less than optimal. > Is there a function or series of functions I can call to somewhat > cleanly shutdown py.test in this situation. maybe. > Longer form: > First It is worth noting that we always run py.test with -x, I suspect > that will have some bearing on what follows.. > > The application I'm testing calls a bunch of C code via ctypes. In the > event of certain pathological failures that C code does an internal > error operation equivalent to assert, i.e. print some stuff and then > call exit. > When this happens there is no way to unroll the C stack to get back to > the calling python context and notify py.test of the failure. > What I have managed to do is intercept the error, and from the > intercept use ctypes callback functionality to call into a python > handler. If you have os.fork ... what happens if you run with the "--boxed" option? > It would be nice if I could raise an exception there and use that to > get back to the original python context, ... You can't normally do that with today's Python interpreters. The "--boxed" above looks like the beginning of a clean solution to the problem: Tests are run in a separate process so that reporting is safe from running weirdness. The current --boxed is somewhat limited but it implements this idea. > that, instead it captures, prints and discards the exception before > returning to the C context. > I have sent them an idea I have for how the exception could instead > optionally be propagated past the C and back to python using setjmp / > longjmp, but I'm not really hopeful that that will be a viable > alternative. > So what I would like to do is clean up py.test in the callback. My > ideal outcome would be if I could do some small amount of stuff that > would make py.test act as if it were exiting due to a failed test > while running with -x. > Is this line of approach at all possible? or does it violate the > py.test structure too much to be workable? Actually i'd like to improve on the idea of running tests and reporting about it in separate processes. Btw, the next level of distributed testing i see in the dynamic setup of environments (via virtualenv or buildout) and running tests there, reporting back to console. best, holger From fdg001 at gmx.net Fri Nov 6 11:36:18 2009 From: fdg001 at gmx.net (Frederik Dohr) Date: Fri, 06 Nov 2009 10:36:18 +0000 Subject: [py-dev] using funcargs for setup/teardown In-Reply-To: <20091026111945.GD29141@trillke.net> References: <4ADE0957.7010005@gmx.net> <20091020210933.GG29141@trillke.net> <4AE45A49.70207@gmx.net> <4AE498C5.4000304@gmx.net> <20091026111945.GD29141@trillke.net> Message-ID: <4AF3FC22.2060906@gmx.net> >> http://gist.github.com/214495 > > not sure i see the need for funcargs with this example - > why don't you use the "traditional" setup_module/teardown_module I agree that it seems like overkill. (That's in part intentional though, due to it being a learning experience.) Yet I liked the idea of having a decorator for expressiveness/salience. Also, I was under the impression that {setup,teardown}_{module,class} were deprecated (cf. Philippe's message), with funcargs generally being the preferred option. Since I need setup and teardown for each individual test function, setup_module doesn't help me much - which leaves me with grouping tests into a class. I'm not entirely happy with that though, because it seems a little boilerplatey and potentially makes for weird grouping of tests. This led me to the option of a decorator wrapping the respective test function in a "try: test(); finally: teardown()" - which worked fine, but seemed to obscure the traceback on failures. For my main project, I've realized that I don't actually need teardown. Instead, I use a function call (could perhaps become a decorator) at the _beginning_ of each test which resets the environment (e.g. erasing the data store). This has the added benefit of leaving inspectable leftovers after the last test (handy with py.test -x). I realize this tabula rasa approach might not be suitable for all projects though. -- F. From holger at merlinux.eu Fri Nov 6 12:34:51 2009 From: holger at merlinux.eu (holger krekel) Date: Fri, 6 Nov 2009 12:34:51 +0100 Subject: [py-dev] using funcargs for setup/teardown In-Reply-To: <4AF3FC22.2060906@gmx.net> References: <4ADE0957.7010005@gmx.net> <20091020210933.GG29141@trillke.net> <4AE45A49.70207@gmx.net> <4AE498C5.4000304@gmx.net> <20091026111945.GD29141@trillke.net> <4AF3FC22.2060906@gmx.net> Message-ID: <20091106113450.GG29141@trillke.net> Hi Frederik, On Fri, Nov 06, 2009 at 10:36 +0000, Frederik Dohr wrote: > >> http://gist.github.com/214495 > > > > not sure i see the need for funcargs with this example - > > why don't you use the "traditional" setup_module/teardown_module > > I agree that it seems like overkill. (That's in part intentional though, > due to it being a learning experience.) > Yet I liked the idea of having a decorator for expressiveness/salience. > > Also, I was under the impression that {setup,teardown}_{module,class} > were deprecated (cf. Philippe's message), with funcargs generally being > the preferred option. I actually considered deprecation but refrained from it. > Since I need setup and teardown for each individual test function, > setup_module doesn't help me much - which leaves me with grouping tests > into a class. I'm not entirely happy with that though, because it seems > a little boilerplatey and potentially makes for weird grouping of tests. > > This led me to the option of a decorator wrapping the respective test > function in a "try: test(); finally: teardown()" - which worked fine, > but seemed to obscure the traceback on failures. I see. > For my main project, I've realized that I don't actually need teardown. > Instead, I use a function call (could perhaps become a decorator) at the > _beginning_ of each test which resets the environment (e.g. erasing the > data store). This has the added benefit of leaving inspectable leftovers > after the last test (handy with py.test -x). > I realize this tabula rasa approach might not be suitable for all > projects though. In any case it's fine to implement custom testing schemes. For that matter you could automate invocation of a module-level "reset_test_environment" by writing down something like this: #./conftest.py # basically a local project-specific plugin import py def pytest_runtest_setup(item): if not isinstance(item, py.test.collect.Function): return # there are js/doc/other tests modnode = item.getparent(py.test.collect.Module) module = modnode.obj # all python collect nodes have 'obj' reset = hasattr('module', 'reset_test_environment', None) if reset is not None: reset(...) # pass custom parameters You can also pick up per-function parameters which you can set with with a decorator or maybe re-using the improved py.test.mark decorator which already works nicely with Python3. HTH, holger From py-dev at tolomea.com Sun Nov 8 23:45:20 2009 From: py-dev at tolomea.com (Gordon Wrigley) Date: Mon, 9 Nov 2009 09:45:20 +1100 Subject: [py-dev] unclean shutdown of py.test In-Reply-To: <20091106105315.GF29141@trillke.net> References: <20091106105315.GF29141@trillke.net> Message-ID: > If you have os.fork ... what happens if you run with the "--boxed" option? While py.test seems to come down somewhat cleanly I get no output capturing at all, even with -s? is this expected behavior? I forgot to mention this in the original email but the main reason I want py.test to come down cleanly is so I can get at the captured output. G From py-dev at tolomea.com Mon Nov 9 00:48:51 2009 From: py-dev at tolomea.com (Gordon Wrigley) Date: Mon, 9 Nov 2009 10:48:51 +1100 Subject: [py-dev] unclean shutdown of py.test In-Reply-To: References: <20091106105315.GF29141@trillke.net> Message-ID: Actually it would probably be more accurate to say that all output is captured. And so I don't see any of my output, both when the test fails and when I run with -s. On Mon, Nov 9, 2009 at 9:45 AM, Gordon Wrigley wrote: >> If you have os.fork ... what happens if you run with the "--boxed" option? > > While py.test seems to come down somewhat cleanly I get no output > capturing at all, even with -s? is this expected behavior? > I forgot to mention this in the original email but the main reason I > want py.test to come down cleanly is so I can get at the captured > output. > > G > From holger at merlinux.eu Mon Nov 9 09:33:53 2009 From: holger at merlinux.eu (holger krekel) Date: Mon, 9 Nov 2009 09:33:53 +0100 Subject: [py-dev] unclean shutdown of py.test In-Reply-To: References: <20091106105315.GF29141@trillke.net> Message-ID: <20091109083353.GA29141@trillke.net> Hi Gordon, On Mon, Nov 09, 2009 at 10:48 +1100, Gordon Wrigley wrote: > Actually it would probably be more accurate to say that all output is > captured. And so I don't see any of my output, both when the test > fails and when I run with -s. Ah, do you mean that --boxed and a crashing process does not give you the output? Currently only a completed test will convey back capturing information but i can see that particularly in the crashing case it's interesting to see the output - could you file an issue for that? cheers, holger > > On Mon, Nov 9, 2009 at 9:45 AM, Gordon Wrigley wrote: > >> If you have os.fork ... what happens if you run with the "--boxed" option? > > > > While py.test seems to come down somewhat cleanly I get no output > > capturing at all, even with -s? is this expected behavior? > > I forgot to mention this in the original email but the main reason I > > want py.test to come down cleanly is so I can get at the captured > > output. > > > > G > > > -- From holger at merlinux.eu Tue Nov 10 00:46:31 2009 From: holger at merlinux.eu (holger krekel) Date: Tue, 10 Nov 2009 00:46:31 +0100 Subject: [py-dev] uploaded a 1.0.0b3 Message-ID: <20091109234631.GI29141@trillke.net> Hi all, just uploaded a new 1.0.0b3 of execnet. see below for the changelog. cheers, holger 1.0.0b3 -------------------------------- * fix EXECNET_DEBUG to work with win32 * add support for serializing longs, sets and frozensets (thanks Benjamin Peterson) * introduce remote_status() method which on the low level gives information about the remote side of a gateway * disallow explicit close in remote_exec situation * perform some more detailed tracing with EXECNET_DEBUG From phil at freehackers.org Tue Nov 10 14:34:39 2009 From: phil at freehackers.org (Philippe Fremy) Date: Tue, 10 Nov 2009 14:34:39 +0100 Subject: [py-dev] using funcargs for setup/teardown In-Reply-To: <20091104175718.GV29141@trillke.net> References: <4ADE0957.7010005@gmx.net> <20091020210933.GG29141@trillke.net> <4AE459EC.805@gmx.net> <20091026102628.GB29141@trillke.net> <4AE7EE2E.1020208@freehackers.org> <20091104175718.GV29141@trillke.net> Message-ID: <4AF96BEF.4050804@freehackers.org> An HTML attachment was scrubbed... URL: From holger at merlinux.eu Tue Nov 10 18:52:11 2009 From: holger at merlinux.eu (holger krekel) Date: Tue, 10 Nov 2009 18:52:11 +0100 Subject: [py-dev] using funcargs for setup/teardown In-Reply-To: <4AF96BEF.4050804@freehackers.org> References: <4ADE0957.7010005@gmx.net> <20091020210933.GG29141@trillke.net> <4AE459EC.805@gmx.net> <20091026102628.GB29141@trillke.net> <4AE7EE2E.1020208@freehackers.org> <20091104175718.GV29141@trillke.net> <4AF96BEF.4050804@freehackers.org> Message-ID: <20091110175211.GN29141@trillke.net> Hi Philippe, could you maybe send your mail again as plain text? i had problems interpreting the html version although i probably got the gist of it. thanks, holger On Tue, Nov 10, 2009 at 14:34 +0100, Philippe Fremy wrote: > > > > > > > holger krekel wrote: >
> 
Hi Philippe,
> 
> On Wed, Oct 28, 2009 at 08:09 +0100, Philippe Fremy wrote:
>
>
> 
holger krekel wrote:
>
>
> 
Hi Frederik!
>   
>
>
> 
However, I've finally figured out why I'm having such a hard time  
> warming up to funcargs: They go way beyond the minimalist simplicity  
> that made me switch to py.test in the first place.
> (I realize there's some Magic to py.test's internals, but that doesn't  
> surface in the API.)
> 
> Funcargs seem like a departure from this principle.
> I understand that some situations demand such complexity, so I'm not  
> arguing against funcargs in general - but for my part, I've managed to  
> keep it simple so far.
>     
>
>
>
> 
I second Frederic here. I chose py.test because of the overall
> simplicity of the framework when you use it. The
> setup_module/setup_class/setup_method without any special magic other
> than using function, class or method names starting with test is really
> really nice. I am kind of worried because I could no longer find them in
> the documentation. I eventually found them in the xUnit documentation
> but that's not where I expected them. I vote for a return of this
> documentation to the main py.test documentation, and moving funcarg
> related documentation to an "advanced test setup" section.
>
>
> 

> I am worried that newcomers will be confused then.  But i see
> the point of "unhiding" the xUnit documentation.  
>
>
> You could present the two approaches in the documentation and > explaining what are the differences.
>
> The simplicity of the setup/teardown model is really a strong asset for > testing. It makes the test code architecture understandable by people > not familiar with py.test . The same can not be said for the funcarg > mechanism...
>
>
>
> 
funcarg seems to be a powerful tool, but really cumbersome to grok. I
> find the magic trick on the naming with __myargument a bit cumbersome. I
> would feel more comfortable with a syntax that mimic the setup/teardown
> used for module, class and methods.
>
>
> 

> It would be easy to also allow explicit registration for factories ala:
> 
>     def pytest_configure(config): 
>         config.add_funcarg_factory(name, factoryfunc) 
> 
> but at the time i choose "convention over configuration" and the  
> "__" because Django also uses a similar way to encode values into
> a function name.  
>
>
> I can understand that, but it does not "speak" to people that are not > famliar with this approach. By using this convention, my impression is > that you are leaving behind the people who are just used to functions, > classes and methods.
>
>
>
>
>
> 
This implies having to call magic methods for setting up objects in 
> global namespaces or 'self' attributes - which not only makes the test 
> harder to understand and refactor IMO. 
>
>
> 
I do not agree here. If I have one class with 30 unit test methods, it's
> easier to setup/teardown the test parameters in two methods for the
> whole class than modifying 30 test methods to add funcargs arguments.
>
>
> 

> agreed, i am considering introducing a new hook:
> 
>     def pytest_pyfunc_setup(request):
>         request.cached_setup(setupfunc, teardownfunc, scope="directory")
> 
> and this hook would be called for each python test function.  Here
> we call the cached_setup helper to help us manage setup/teardown scopes. 
> (see http://tinyurl.com/yfw82l5 for request object attributes
> which you could pass into your setup/teardown func)
> 
> It would also mean you could write down:
> 
>     class TestGroup:
>         parameter = 3
>         def test_method(self):
>             assert self.parametrized_obj  ...
> 
> and implement a pytest_pyfunc_setup() to set a "self.someobj" 
> according to the class-specified parameter. 
> 
> Would this make sense to you?  
>
>
> Honestly, I am not sure I get it all.  Can you sketch full examples ?
>
> By the way, is it possible to dynamically update the py.test namespace > ? There is the initial hook py_test_namespace() but it's only called > once. A py.test.update_namespace() would be welcome in my case.
>
>
>
> 
So, while I agree that funcargs certainly has potential, I think you
> should not force it onto the user and should really stress the two ways
> of setting up per class or per method parameters.
>
>
> 

> ok, I'll see to work on the docs a bit. 
>  
>
>
> 
And it would be really nice to figure out a syntax for funcarg that is
> more in the setup/teardown fashion.
> 
> I don't see much gain of using :
> 
> def pytest_funcarg__mysetup(request):
>     return MySetup(request)
> 
> class TestClass:
>     def test_function(self, mysetup):
>         conn = mysetup.getsshconnection()
>         # work with conn
> 
> 
> instead of :
> 
> class TestClass:
>     def setup_class( c ):
>         c.mysetup = MySetup()
> 
>     def test_function(self):
>         conn = c.mysetup.getsshconnection()
>         # work with conn
> 
> 
> The programming style is different, the second one is traditional OO and
> will be familiar to anybody coming from C++, Java or other OO world. The
> second one is playing more on the python capabilities. I don't find it
> more readable.
>
>
> 

> If you have setup_module + setup_class + setup_method (+ subclassing!) 
> layers of setup and teardown, things get a bit harder to understand. 
> In >3000 test projects i know off this led to complexity and 
> refactoring difficulties. 
>
>
> I haven't reached for > 3000 test cases, so I can not comment on > that. But for small projects like the one I am working with (around a > few hundreds tests), the setup/teardown works very well.
>
>
> 
One can argue about the advantage that each test function can take a
> different parameter. While true, in my testing experience, I haven't
> seen a pattern with many different tests taking many different
> parameters. I usually have groups of 5 to 10 tests taking one kind of
> parameter. If they need to take another kind of parameter, I will put
> them in a different test class.
>
>
> 

> These days i sometimes group tests by feature rather than by setup-parameters. 
> E.g. i put two unit-tests and one functional test into a "TestFeature" class. 
>
>
> Then you need a different setup depending on which test you are running > indeed. But you can already do that by discriminating on the method > argument of the setup_method(), no ?
>
>
> 
So, in my opinion, using funcargs for regular cases is a matter of style.
> 
> I do see a big value in funcargs for the parameterized tests. Running
> the same test over and over with different parameter is a really nice
> feature. I think you should stress it more in the documentation.
>
>
> 

> makes sense. thanks a lot for your feedback! 
> 
> best,
> holger
> 
>
>
>
> > > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev -- From phil at freehackers.org Tue Nov 10 21:06:23 2009 From: phil at freehackers.org (Philippe Fremy) Date: Tue, 10 Nov 2009 21:06:23 +0100 Subject: [py-dev] using funcargs for setup/teardown In-Reply-To: <20091104175718.GV29141@trillke.net> References: <4ADE0957.7010005@gmx.net> <20091020210933.GG29141@trillke.net> <4AE459EC.805@gmx.net> <20091026102628.GB29141@trillke.net> <4AE7EE2E.1020208@freehackers.org> <20091104175718.GV29141@trillke.net> Message-ID: <4AF9C7BF.1010309@freehackers.org> Hi Holger, Same mail again, this time in plain good old text. Sorry for that, now that I'm using only html for professional email, I tend to forget my good manners for open source projects. holger krekel wrote: > Hi Philippe, > > On Wed, Oct 28, 2009 at 08:09 +0100, Philippe Fremy wrote: > >> holger krekel wrote: >> >>> Hi Frederik! >>> >>> >>>> However, I've finally figured out why I'm having such a hard >>>> time warming up to funcargs: They go way beyond the minimalist >>>> simplicity that made me switch to py.test in the first place. >>>> (I realize there's some Magic to py.test's internals, but that >>>> doesn't surface in the API.) >>>> >>>> Funcargs seem like a departure from this principle. I >>>> understand that some situations demand such complexity, so I'm >>>> not arguing against funcargs in general - but for my part, >>>> I've managed to keep it simple so far. >>>> >>>> >> I second Frederic here. I chose py.test because of the overall >> simplicity of the framework when you use it. The >> setup_module/setup_class/setup_method without any special magic >> other than using function, class or method names starting with test >> is really really nice. I am kind of worried because I could no >> longer find them in the documentation. I eventually found them in >> the xUnit documentation but that's not where I expected them. I >> vote for a return of this documentation to the main py.test >> documentation, and moving funcarg related documentation to an >> "advanced test setup" section. >> > > I am worried that newcomers will be confused then. But i see the > point of "unhiding" the xUnit documentation. > You could present the two approaches in the documentation and explaining what are the differences. The simplicity of the setup/teardown model is really a strong asset for testing. It makes the test code architecture understandable by people not familiar with py.test . The same can not be said for the funcarg mechanism... >> funcarg seems to be a powerful tool, but really cumbersome to grok. >> I find the magic trick on the naming with __myargument a bit >> cumbersome. I would feel more comfortable with a syntax that mimic >> the setup/teardown used for module, class and methods. >> > > It would be easy to also allow explicit registration for factories > ala: > > def pytest_configure(config): config.add_funcarg_factory(name, > factoryfunc) > > but at the time i choose "convention over configuration" and the "__" > because Django also uses a similar way to encode values into a > function name. > I can understand that, but it does not "speak" to people that are not famliar with this approach. By using this convention, my impression is that you are leaving behind the people who are just used to functions, classes and methods. >>> This implies having to call magic methods for setting up objects >>> in global namespaces or 'self' attributes - which not only makes >>> the test harder to understand and refactor IMO. >>> >> I do not agree here. If I have one class with 30 unit test methods, >> it's easier to setup/teardown the test parameters in two methods >> for the whole class than modifying 30 test methods to add funcargs >> arguments. >> > > agreed, i am considering introducing a new hook: > > def pytest_pyfunc_setup(request): request.cached_setup(setupfunc, > teardownfunc, scope="directory") > > and this hook would be called for each python test function. Here we > call the cached_setup helper to help us manage setup/teardown > scopes. (see http://tinyurl.com/yfw82l5 for request object attributes > which you could pass into your setup/teardown func) > > It would also mean you could write down: > > class TestGroup: parameter = 3 def test_method(self): assert > self.parametrized_obj ... > > and implement a pytest_pyfunc_setup() to set a "self.someobj" > according to the class-specified parameter. > > Would this make sense to you? > Honestly, I am not sure I get it all. Can you sketch full examples ? By the way, is it possible to dynamically update the py.test namespace ? There is the initial hook py_test_namespace() but it's only called once. A py.test.update_namespace() would be welcome in my case. >> So, while I agree that funcargs certainly has potential, I think >> you should not force it onto the user and should really stress the >> two ways of setting up per class or per method parameters. >> > > ok, I'll see to work on the docs a bit. > > >> And it would be really nice to figure out a syntax for funcarg that >> is more in the setup/teardown fashion. >> >> I don't see much gain of using : >> >> def pytest_funcarg__mysetup(request): return MySetup(request) >> >> class TestClass: def test_function(self, mysetup): conn = >> mysetup.getsshconnection() # work with conn >> >> >> instead of : >> >> class TestClass: def setup_class( c ): c.mysetup = MySetup() >> >> def test_function(self): conn = c.mysetup.getsshconnection() # work >> with conn >> >> >> The programming style is different, the second one is traditional >> OO and will be familiar to anybody coming from C++, Java or other >> OO world. The second one is playing more on the python >> capabilities. I don't find it more readable. >> > > If you have setup_module + setup_class + setup_method (+ > subclassing!) layers of setup and teardown, things get a bit harder > to understand. In >3000 test projects i know off this led to > complexity and refactoring difficulties. > I haven't reached for > 3000 test cases, so I can not comment on that. But for small projects like the one I am working with (around a few hundreds tests), the setup/teardown works very well. >> One can argue about the advantage that each test function can take >> a different parameter. While true, in my testing experience, I >> haven't seen a pattern with many different tests taking many >> different parameters. I usually have groups of 5 to 10 tests taking >> one kind of parameter. If they need to take another kind of >> parameter, I will put them in a different test class. >> > > These days i sometimes group tests by feature rather than by > setup-parameters. E.g. i put two unit-tests and one functional test > into a "TestFeature" class. > Then you need a different setup depending on which test you are running indeed. But you can already do that by discriminating on the method argument of the setup_method(), no ? >> So, in my opinion, using funcargs for regular cases is a matter of >> style. >> >> I do see a big value in funcargs for the parameterized tests. >> Running the same test over and over with different parameter is a >> really nice feature. I think you should stress it more in the >> documentation. >> > > makes sense. thanks a lot for your feedback! > > best, holger > > From fdg001 at gmx.net Wed Nov 11 13:11:47 2009 From: fdg001 at gmx.net (Frederik Dohr) Date: Wed, 11 Nov 2009 12:11:47 +0000 Subject: [py-dev] virtual namespace for plugins Message-ID: <4AFAAA03.7010209@gmx.net> Holger and I had a conversation about virtual namespaces on IRC. We're using this to have a common namespace for plugins*, and Holger expressed interest in using this approach as well. The goal is to use a common package name (here: "foo") for multiple independent modules or packages ("alpha" and "bravo" - to be accessed as foo.alpha and foo.bravo, respectively). It's actually rather simple: The individual components must be contained in a directory named after the virtual namespace, with an __init__.py containing nothing but the following statement: __import__("pkg_resources").declare_namespace(__name__) To make Python aware of the local package (e.g. for running tests out of the repository), something like the following is required in a root-level __init__.py: path = os.path.abspath(VIRTUAL_NAMESPACE) sys.modules[VIRTUAL_NAMESPACE].__dict__["__path__"].insert(0, path) This is illustrated here: http://gist.github.com/231862 Disclaimer: I'm far from being an expert, just sharing what we've learned so far (it's an ongoing effort). -- F. * http://cdent.tumblr.com/post/216241761/python-namespace-packages-for-tiddlyweb From fdg001 at gmx.net Thu Nov 12 19:23:43 2009 From: fdg001 at gmx.net (Frederik Dohr) Date: Thu, 12 Nov 2009 18:23:43 +0000 Subject: [py-dev] virtual namespace for plugins In-Reply-To: <4AFAAA03.7010209@gmx.net> References: <4AFAAA03.7010209@gmx.net> Message-ID: <4AFC52AF.3090401@gmx.net> > To make Python aware of the local package (e.g. for running tests out of > the repository), something like the following is required in a > root-level __init__.py: > path = os.path.abspath(VIRTUAL_NAMESPACE) > sys.modules[VIRTUAL_NAMESPACE].__dict__["__path__"].insert(0, path) It appears my testing was flawed; this __init__.py is not being picked up automatically after all (which seemed odd in the first place), at least not by setup.py. So an explicit "import __init__" is required - in which case that module should be renamed, of course. -- F. From holger at merlinux.eu Wed Nov 18 11:53:31 2009 From: holger at merlinux.eu (holger krekel) Date: Wed, 18 Nov 2009 11:53:31 +0100 Subject: [py-dev] 1.1.1 wishes, bugs? Message-ID: <20091118105331.GJ17999@trillke.net> Hi, i shortly plan to put out a 1.1.1 fixing some small issues. anything i should look at/consider for the release? If it's small annoyances, just post here, no need to open "issues". best, holger From lac at openend.se Wed Nov 18 14:22:00 2009 From: lac at openend.se (Laura Creighton) Date: Wed, 18 Nov 2009 14:22:00 +0100 Subject: [py-dev] 1.1.1 wishes, bugs? In-Reply-To: Message from holger krekel of "Wed, 18 Nov 2009 11:53:31 +0100." <20091118105331.GJ17999@trillke.net> References: <20091118105331.GJ17999@trillke.net> Message-ID: <200911181322.nAIDM0v6022185@theraft.openend.se> In a message of Wed, 18 Nov 2009 11:53:31 +0100, holger krekel writes: >Hi, > >i shortly plan to put out a 1.1.1 fixing some small issues. >anything i should look at/consider for the release? >If it's small annoyances, just post here, no need >to open "issues". > >best, >holger Small annoyance: Have py.test --help print the message that --report=skipped is how to get more information about skipped tests. Laura From holger at merlinux.eu Tue Nov 24 17:49:50 2009 From: holger at merlinux.eu (holger krekel) Date: Tue, 24 Nov 2009 17:49:50 +0100 Subject: [py-dev] execnet-1.0.0 released Message-ID: <20091124164950.GI17999@trillke.net> Hi all, execnet-1.0.0 (compared to 1.0.0b3) has bug fixes, new tested examples and introduces execnet.Group for managing a dynamic bunch of hosts. See the improved docs http://codespeak.net/execnet/ and below the changelog fragment. cheers, holger 1.0.0 compared to 1.0.0b3 -------------------------------- * introduce execnet.Group for managing gateway creation and termination. Introduce execnet.default_group through which all "global" calls are routed. cleanup gateway termination. All Gateways get an id through which they can be retrieved from a group object. * deprecate execnet.XYZGateway in favour of direct makegateway() calls. * refine socketserver-examples, experimentally introduce a way to indirectly setup a socket server ("installvia") through a gateway url. * refine and automatically test documentation examples From holger at merlinux.eu Tue Nov 24 18:12:49 2009 From: holger at merlinux.eu (holger krekel) Date: Tue, 24 Nov 2009 18:12:49 +0100 Subject: [py-dev] py.test-1.1.1: compat fixes, setuptools-plugin registration Message-ID: <20091124171249.GK17999@trillke.net> py.test/pylib 1.1.1: bugfix release, setuptools plugin registration -------------------------------------------------------------------------------- This is a compatibility fixing release of pylib/py.test to work better with previous 1.0.x test code bases. It also contains fixes and changes to work with `execnet>=1.0.0`_ to provide distributed testing and looponfailing testing modes. py-1.1.1 moreover introduces a new mechanism for registering plugins via setuptools. What is pylib/py.test? ----------------------- py.test is an advanced automated testing tool working with Python2, Python3 and Jython versions on all major operating systems. It has an extensive plugin architecture and can run many existing common Python test suites without modification. Moreover, it offers some unique features not found in other testing tools. See http://pytest.org for more info. The pylib provides local and svn filesystem Path objects installs some developer-oriented command line tools. See http://pylib.org for more info. thanks to all who helped and gave feedback, have fun, holger (http://twitter.com/hpk42) .. _`execnet>=1.0.0`: http://codespeak.net/execnet Changes between 1.1.1 and 1.1.0 ===================================== - introduce automatic plugin registration via 'pytest11' entrypoints via setuptools' pkg_resources.iter_entry_points - fix py.test dist-testing to work with execnet >= 1.0.0b4 - re-introduce py.test.cmdline.main() for better backward compatibility - svn paths: fix a bug with path.check(versioned=True) for svn paths, allow '%' in svn paths, make svnwc.update() default to interactive mode like in 1.0.x and add svnwc.update(interactive=False) to inhibit interaction. - refine distributed tarball to contain test and no pyc files - try harder to have deprecation warnings for py.compat.* accesses report a correct location From holger at merlinux.eu Sat Nov 28 01:34:08 2009 From: holger at merlinux.eu (holger krekel) Date: Sat, 28 Nov 2009 01:34:08 +0100 Subject: [py-dev] ciss-0.1: code-centered issue tracking Message-ID: <20091128003408.GR29390@trillke.net> Hi all, just released ciss-0.1, my attempt at (what i call) code-centered issue tracking. ciss is: - a command line tool for managing your ISSUES.txt - code-centered: associate issues to files in your project - extensible: assign tags for status/milestone/custom usage. - ueber-powerful issue editing: your text editor! - well tested (more tests than code) If that appeals to you somewhat, check it out: http://codespeak.net/ciss And in case you wonder, why i don't use pitz or ditz? For one, I don't want to edit my issues from an interactive Python prompt, sorry. Second, they create directories and pickle-files and whatnot - i just want to keep a plain text file for a number of projects and using a text editor and a nice cmdline tool is the fastest way i can imagine. Oh, and I do see use in web-based issue interaction but keeping issues close-to-code (tm) and easy-to-edit (tm) is just so much more fun. Besides, i believe 'ciss' could learn to round-trip with existing trackers. anyway, have fun, holger From fdg001 at gmx.net Sat Nov 28 12:33:51 2009 From: fdg001 at gmx.net (Frederik Dohr) Date: Sat, 28 Nov 2009 11:33:51 +0000 Subject: [py-dev] ciss-0.1: code-centered issue tracking In-Reply-To: <20091128003408.GR29390@trillke.net> References: <20091128003408.GR29390@trillke.net> Message-ID: <4B110A9F.4050304@gmx.net> > just released ciss-0.1, my attempt at (what i call) > code-centered issue tracking. This certainly looks interesting! (Note: I missed the tutorial link at first, maybe that should be more prominent?) I'm a big proponent of todo.sh (http://todotxt.com), and this fits into the same category - and could even be useful beyond coding. FWIW, todo.sh supports nanoformats like @context and +project (tags and categories, essentially). Perhaps that's worth considering (even though ciss uses multi-line items). > i believe 'ciss' could learn to round-trip with existing trackers That would be most useful. Thanks for sharing! -- F.