From holger at merlinux.eu Wed Sep 1 12:55:54 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 1 Sep 2010 12:55:54 +0200 Subject: [py-dev] Greenlet In-Reply-To: References: Message-ID: <20100901105553.GC1914@trillke.net> Hi Richard, On Sat, Aug 21, 2010 at 09:20 +0800, Richard Tew wrote: > Hi, > > In the past, I believe py included the greenlet c extension source > code. I am not sure that you do anymore, or how this affects you now > that it may only relate to older releases of your source code, but I > would like to mention that the py library as far as I can tell > effectively relicensed Python licensed code under the MIT license. > Please correct me if I am wrong. That is correct but there also was a note that parts of the lib may have a different license ... > http://codespeak.net/svn/py/branch/py-compat-2.5.2/py/c-extension/greenlet/ It should have been mentioned there, not sure it was in the final release ... probably not :/ > The files sourced from Stackless presumably by Christian for use with > Armin in the creation of greenlet, were contributed to Stackless and > subject to contributions from multiple parties, under the Python > license. I am pretty sure that Christian did not, and was more than > likely unable to relicense them to the MIT license on behalf of those > parties. I guess so as well. OTOH i think MIT and Python license are not that far away from each other. In any case, greenlets are not part of the py lib for a long time now so i don't intend to do anything on that front. > Anyway, I've brought this up with the greenlet project here: > > http://bitbucket.org/ambroff/greenlet/ > > And since that was spun off the py lib, I thought I would mention it here. Thanks for the info. I agree to your comment on the issue there that it's all a bit unfortunate to concern ourselves and I also guess it makes sense to just say that greenlet is MIT *and* Python-licensed for now. So everybody who needs to care should make sure they comply to both licenses (which is not really hard and there hardly is anyone likely to ever challenge usages). just my 2cent and sorry for probably having played a part in the unprecise original license tracking. best, holger From arigo at tunes.org Wed Sep 1 15:43:01 2010 From: arigo at tunes.org (Armin Rigo) Date: Wed, 1 Sep 2010 15:43:01 +0200 Subject: [py-dev] Greenlet In-Reply-To: <20100901105553.GC1914@trillke.net> References: <20100901105553.GC1914@trillke.net> Message-ID: Hi, I have clarified the license of greenlets as found in its own svn repository subtree at http://codespeak.net/svn/greenlet/trunk/. Greenlet is no longer part of the py lib, so I don't want to go into any licensing complication; I just adopted the same license as Stackless Python. Note that most (but not all?) of the code might also be available under the MIT license from the time where it was part of the py lib; that's fine by me if you care enough to dig that out of the history. I have no control over http://bitbucket.org/ambroff/greenlet/ so if that's what you are using, you need to check with them. Armin From holger at merlinux.eu Wed Sep 1 16:00:36 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 1 Sep 2010 16:00:36 +0200 Subject: [py-dev] py.test doesn't like functools.partial wrappers In-Reply-To: <34CD30EDA5E4CB4AAA2501EB1ED319E102D705B6@charlie.diasemi.com> References: <34CD30EDA5E4CB4AAA2501EB1ED319E102D705B6@charlie.diasemi.com> Message-ID: <20100901140036.GE1914@trillke.net> Hi Florian, On Fri, Aug 20, 2010 at 15:49 +0200, Florian Bauer wrote: > Hi all, > > I'm trying to make an existing testsuite py.test compatible. > At the moment, we use nose as test runner. I stumbled upon the following > (simplified example). > > from functools import partial > > def forall_cases(cases): > def decorate(testfn): > def gen(): > for case in cases: > description = case[0] > args = case[1:] > func = partial(testfn, *args) > func.description = description > yield func, > gen.__name__= 'test_%s_' % testfn.__name__ > # > # inject the generator into the module testfn came from > # > gen.__module__ = testfn.__module__ > return gen > return decorate > > > @forall_cases([['case1', 1, 1], ['case2', 2, 2],['case3', 3, 4]]) def > test_equal(a, b): > assert a == b Hum, is it an option to rather use a different decorator that makes use of py.test's more general parametrized testing feature? See here: http://tetamap.wordpress.com/2009/05/13/parametrizing-python-tests-generalized/ for some general examples. In your case you would need to 1) define @forallcases as a simple marker, e.g.: forallcases = py.test.mark.forallcases 2) implement a pytest_generate_tests hooks that checks for the "forallcases" decorator and generates the test function invocations. The second part can just live in a conftest.py file and does not conflict with the nose-way of running tests. For the first you could try to just do something like def pytest_configure(config): import module_that_contains_forallcases module_that_contains_forallcases.forallcases = py.test.mark.forallcases If the latter lives in a conftest.py file at the root of your project or at the root of all test modules it should be executed before any test module is imported (which might import and use the forallcases decorator). > If I run this with nosetests, I get 3 test cases, 2 pass and 1 fail. > > py.test tries to call inspect.getsource on the partial object, which > results in a TypeError: > > $ py.test -v > ... > INTERNALERROR> > INTERNALERROR> object = > INTERNALERROR> > INTERNALERROR> def getfile(object): > INTERNALERROR> """Work out which source or compiled file an > INTERNALERROR> object was d > efined in.""" Can you post or attach a full traceback? I agree that a) it would be nice if nose and py.test would be more compatible here b) the error is obscure. OTOH i consider yield-based parametrization as a kind of dead-end. It certainly causes a more complex implementation for the testing machinery and can not provide features/reporting as cleanly as the above mentioned parametrized testing. > ... > INTERNALERROR> > raise TypeError('arg is not a module, class, > method, ' > INTERNALERROR> 'function, traceback, frame, or > INTERNALERROR> code obje > ct') > INTERNALERROR> E TypeError: arg is not a module, class, method, > function, > traceback, frame, or code object > INTERNALERROR> > INTERNALERROR> object = > INTERNALERROR> > INTERNALERROR> c:\Python26\lib\inspect.py:418: TypeError > > This seems to be a documented limitation of inspect.getsource > (functools.partial returns a partial object, not a function). > I can fix this by replacing functools.partial with code given as roughly > equivalent in the python documentation: > > def partial(func, *args, **keywords): > def newfunc(*fargs, **fkeywords): > newkeywords = keywords.copy() > newkeywords.update(fkeywords) > return func(*(args + fargs), **newkeywords) > newfunc.func = func > newfunc.args = args > newfunc.keywords = keywords > return newfunc > > But I was certainly not expecting this. The default behavior of py.test > is even more puzzling, as it fails silently: > > fbauer at MN-ENG-LT-FB ~/My Documents/Software/pytestbug1 $ py.test > ============================= test session starts > ============================= platform win32 -- Python 2.6.5 -- > pytest-1.3.3 test path 1: c:\docume~1\fbauer\My > Documents\Software\pytestbug1 > > test_case.py .. > > In my actual code, the forall_cases decorator lives in a utils module. > Is there a conditional flag that can tell me whether I'm running under > py.test? Then I could start experimenting with parametric tests using > funcargs, while keeping the test suite runnable with nose at the moment. > I still have a bug in the test suite, as I have tests passing under nose > that fail under py.test... There is no official way to detect if code is running within a test session. You might do something like: # contents of conftest.py def pytest_configure(config): import some_app_module some_app_module._testing = True and then do a "hasattr(some_app_module, '_testing')" check or so. However, most people don't consider it good practise to have application level code become aware if it's being tested or not. Instead people use monkeypatching or dependency-injection to modify app behaviour. HTH, holger From holger at merlinux.eu Wed Sep 1 16:40:34 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 1 Sep 2010 16:40:34 +0200 Subject: [py-dev] [proposal] extended information collection In-Reply-To: <1282042493.12940.16.camel@klappe2> References: <1282042493.12940.16.camel@klappe2> Message-ID: <20100901144034.GF1914@trillke.net> On Tue, Aug 17, 2010 at 12:54 +0200, Ronny Pfannschmidt wrote: > Recently i noticed a new need. > > The collecting/asserting various kinds of data during a testrun. > > I noticed the following (partially hpyothetical) cases in various > projects > > * pip records file changes to the virtualenv it makes for testing > * pypy records jit compile results/dumps > * jinja2 should record asts and the compile results of templates it > tests > * the logcapture extension > * the build-in io captureing > > i think i would be helpful/practical to have a common way to record > those things and navigate them in extended test reporters Not sure how you can uniformly present a user-interface to navigate such data. > adding propper timing information to lines/data could help even more > (for having a common timeline of different event sources) > > since this topic is rather hard to manage i hope for some dicussion on > the needs others see, as well as if/how this could add more to the > testing Maybe a simple storage & query mechanism for test-run related data could help. I never got around to implement this. It would provide useful to allow features like "run the last failing test from a previous py.test cmdline invocation" and to generate html reports from subsequent revisions of a project where we do a test-run for each revision and later query the result storage. This should be easily implementable as a plugin. best, holger From Florian.Bauer at diasemi.com Thu Sep 2 10:50:18 2010 From: Florian.Bauer at diasemi.com (Florian Bauer) Date: Thu, 2 Sep 2010 10:50:18 +0200 Subject: [py-dev] FW: py.test doesn't like functools.partial wrappers Message-ID: <34CD30EDA5E4CB4AAA2501EB1ED319E102FADE64@charlie.diasemi.com> ... Aargh, forgot to reply to list... -----Original Message----- From: Florian Bauer Sent: Mittwoch, 1. September 2010 16:55 To: holger krekel Subject: RE: [py-dev] py.test doesn't like functools.partial wrappers Hi Holger, > -----Original Message----- > From: holger krekel [mailto:holger at merlinux.eu] > > On Fri, Aug 20, 2010 at 15:49 +0200, Florian Bauer wrote: > > Hi all, > > > > I'm trying to make an existing testsuite py.test compatible. > > At the moment, we use nose as test runner. I stumbled upon the > > following (simplified example). > > > > from functools import partial > > > > def forall_cases(cases): > > def decorate(testfn): > > def gen(): > > for case in cases: > > description = case[0] > > args = case[1:] > > func = partial(testfn, *args) > > func.description = description > > yield func, > > gen.__name__= 'test_%s_' % testfn.__name__ > > # > > # inject the generator into the module testfn came from > > # > > gen.__module__ = testfn.__module__ > > return gen > > return decorate > > > > > > @forall_cases([['case1', 1, 1], ['case2', 2, 2],['case3', > 3, 4]]) def > > test_equal(a, b): > > assert a == b > > Hum, is it an option to rather use a different decorator that makes > use of py.test's more general parametrized testing feature? > See here: Yes, definetely. I would like to leave this decorator untouched for now (keeping nose working), and would be quite happy to have py.test using a different decorator implementation. I will try this! > > http://tetamap.wordpress.com/2009/05/13/parametrizing-python-t > ests-generalized/ > > for some general examples. In your case you would need to > 1) define @forallcases as a simple marker, e.g.: > forallcases = py.test.mark.forallcases > > 2) implement a pytest_generate_tests hooks that checks for the > "forallcases" > decorator and generates the test function invocations. > > The second part can just live in a conftest.py file and does not > conflict with the nose-way of running tests. > > For the first you could try to just do something like > > def pytest_configure(config): > import module_that_contains_forallcases > module_that_contains_forallcases.forallcases = > py.test.mark.forallcases > > If the latter lives in a conftest.py file at the root of your project > or at the root of all test modules it should be executed before any > test module is imported (which might import and use the forallcases > decorator). I will play around with this, thanks! > > If I run this with nosetests, I get 3 test cases, 2 pass and 1 fail. > > > > py.test tries to call inspect.getsource on the partial > object, which > > results in a TypeError: > > > > $ py.test -v > > ... > > INTERNALERROR> > > INTERNALERROR> object = > > INTERNALERROR> > > INTERNALERROR> def getfile(object): > > INTERNALERROR> """Work out which source or compiled file an > > INTERNALERROR> object was d > > efined in.""" > > Can you post or attach a full traceback? > I agree that a) it would be nice if nose and py.test would be more > compatible here b) the error is obscure. OTOH i consider yield-based > parametrization as a kind of dead-end. > It certainly causes a more complex implementation for the testing > machinery and can not provide features/reporting as cleanly as the > above mentioned parametrized testing. I attached a full traceback. I agree with you on the yield based-parametrization. That's why I would like to move away from it and am experimenting with py.test. My secret goal is to implement something like Haskell's quickcheck. Generation of random test data is the easy part. The fun begins when you start to think about the different reporting needs a test might have in that case ('generated 10000 test cases, 97% pass, 14% trivial' for example). But first I have to migrate the current test suite... > > Is there a conditional flag that can tell me whether I'm > running under > > py.test? Then I could start experimenting with parametric > tests using > > funcargs, while keeping the test suite runnable with nose > at the moment. > > I still have a bug in the test suite, as I have tests > passing under nose > > that fail under py.test... > > There is no official way to detect if code is running within a test > session. > You might do something like: > > # contents of conftest.py > def pytest_configure(config): > import some_app_module > some_app_module._testing = True > > and then do a "hasattr(some_app_module, '_testing')" check or so. > However, most people don't consider it good practise to have > application > level code become aware if it's being tested or not. Instead > people use monkeypatching or dependency-injection to modify app > behaviour. My use case was actually to check in my test suite whether it's running under nose or under py.test. I think your (and Ronny's) suggestion to use the conftest mechanism provides a much cleaner solution. Thanks, Florian _______________________________________________________________________________________ Dialog Semiconductor GmbH Neue Str. 95 D-73230 Kirchheim Managing Director: Dr. Jalal Bagherli Chairman of the Supervisory Board: Gregorio Reyes Commercial register: Amtsgericht Stuttgart: HRB 231181 UST-ID-Nr. DE 811121668 Legal Disclaimer: This e-mail communication (and any attachment/s) is confidential and contains proprietary information, some or all of which may be legally privileged. It is intended solely for the use of the individual or entity to which it is addressed. Access to this email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. -------------- next part -------------- A non-text attachment was scrubbed... Name: traceback Type: application/octet-stream Size: 23326 bytes Desc: traceback URL: From holger at merlinux.eu Thu Sep 2 11:27:22 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 2 Sep 2010 11:27:22 +0200 Subject: [py-dev] Advanced assert equal In-Reply-To: <20100816135154.GA24594@laurie.devork.be> References: <20100816002557.GA12335@laurie.devork.be> <1281958153.12940.5.camel@klappe2> <20100816135154.GA24594@laurie.devork.be> Message-ID: <20100902092722.GG1914@trillke.net> Hi Floris, sidenote: the original implementation was done by Armin Rigo and i am not much more accustomed to the assert-interpretation code than you are. The other guy having done bits there is Benjamin - still haven't got hold of him on this issue and your mail. But I discussed with Benjamin introducing a more general API that would have the AST-transformation call back into a generic pytest-hook if an assertion fails. So one could customize "hasattr(x, y)" or "x in y" or "x == y == z" etc. and all such representation code would move to the py/_plugin/pytest_assertion.py plugin. That being said going for just comparisons right now is fine, let's not have more general thoughts hold us up. On Mon, Aug 16, 2010 at 14:51 +0100, Floris Bruynooghe wrote: > On Mon, Aug 16, 2010 at 01:29:13PM +0200, Ronny Pfannschmidt wrote: > > On Mon, 2010-08-16 at 01:25 +0100, Floris Bruynooghe wrote: > > > The attached patch makes compare equal a special case and checks if > > > the two arguments to it are both a list, text or dict and tries to > > > generate a nicer explanation text for them. The patch is more like a > > > proof of concept then a final implementation, I may have done some > > > very strange or silly things as I'm not familiar with the code. It > > > would be great to get feedback, both on the general concept and the > > > actual implementation (particularly note the way I had to hack > > > _format_explanation() in assertion.py). > > > > I think it will be helpful to have some kind of hook to add more > > explain-functions > > Adding hooks should be possible, looking at all the .visit_*() > functions it would seem only one hook is required, unless separate > hooks for each rich compare operator are deemed useful. no, a single one for all comparisons sound fine. > The trickiest bit I think is how to produce multiline explanations. > _format_explanation() concatenates all newlines. Having only special > cases for \n{ and \n} which is used by .visit_Call() to force nested > and indented newlines. In the patch I added \n== for this but a more > general one is probably required, something like \n> or \n~ could work > I guess. This could be completely hidden from the hook however, by > returning a list for each line to be printed, the caller of the hook > would then join these up correctly so that _format_explanation() will > add newlines and indentation correctly. Makes sense to have the caller deal with concatentation. > A possible api for the hook > could be: > > def pytest_assert_compare(op, left, right, left_repr, right_repr): > """Customise compare > > Return None for no custom compare, otherwise return a list of > strings. The strings will be joined by newlines but any newlines > *in* a string will be escaped. > """ > pass > > I guess the reprs are not really necessary if there's an easy way to > make them. It's just that I used them in my original patch. Hum, couldn't custom representations just mean that there is no "history" of where the object comes from? The hook looks otherwise good to me. > Another option is to encapsulate the arguments into an object that > also knows how the builtin types and operators are compared, something > like: > > class CompareItem(object): > def __init__(self, op, left, right, ...): > self.op = op > self.left = left > self.right = right > ... > > def default(self): > if type(self.left) != type(self.right): > return None > if self.op == '==': > if isinstance(self.left, (list, tuple)): > return self.sequence_equal() > elif isinstance(self.left, basestring): > return self.string_equal() > ... > elif self.op == '!=': > ... > > def sequence_equal(self): > pass > > def string_equal(self): > pass > > ... > This would allow pytest_assert_compare() to use those methods as part > of the implementation. I think the other style is more sensible because delegation to "builtin representation styles" a) can happen via plugins and the plugin mechanism b) we could pass a "testrepr" or "builtinrepr" argument to the hook that helps to invoke helpful default machinery. (I guess you are aware that any pytest-hook implementation can always choose to accept less than the available arguments). > There's also the question of who should truncate large amounts of data > (e.g. screenfulls of diffs): the hook itself, the caller of the hook > or _format_explanation()? Probably one of the first two to get rid of > the memory usage as soon as possible. If a hook returns something we should (probably) not further do anything with it in _format_explanation(). And making truncation the repsonsibility of the hook makes sense i think. > > In particular, cause there are many more build-in types to manage, > > and i have at least 2 projects where custom compare-explain is helpfull > > > > another wishlist item i see is the rest of rich compare > > i.e. <, >, <=, >=, != > > Sure, all builtin types and operators should ideally be supported by > default as best as possible. I just started with some I wanted most. I am not sure about the general use cases, from my side: x == y x != y x in y are the interesting ones (with some of the objects being long lists, long strings etc.). so some hook for a "binary" relation makes sense, pytest_assert_binrepr(op) where op could be "==", "in" or "is" etc. Floris, i'd very much like to start using/having improved assertion representations. Happy to review patches or forks for inclusion. best, holger From issues-noreply at bitbucket.org Fri Sep 3 16:23:27 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Fri, 03 Sep 2010 14:23:27 -0000 Subject: [py-dev] New issue 116 in py-trunk: ImportMismatchError or ImportError encountered running py.test --doctest-modules with a package present Message-ID: --- you can reply above this line --- New issue 116: ImportMismatchError or ImportError encountered running py.test --doctest-modules with a package present http://bitbucket.org/hpk42/py-trunk/issue/116/importmismatcherror-or-importerror-encountered-running-pytest-doctest-modules-with-a-package Jason R. Coombs / jaraco on Fri, 3 Sep 2010 16:23:26 +0200: Description: To simply reproduce in py==1.3.3: {{{ mkdir package; touch package/__init__.py; py.test --doctest-modules }}} Results in the attached error messages (different on linux and Windows). running `py.test --doctest-modules package` seems to work around the issue. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From holger at merlinux.eu Fri Sep 3 17:30:34 2010 From: holger at merlinux.eu (holger krekel) Date: Fri, 3 Sep 2010 17:30:34 +0200 Subject: [py-dev] do you mind the issue-postings here? Message-ID: <20100903153034.GE32478@trillke.net> Hi all, am wondering: do you mind seeing new bitbucket issue creations here on the mailing list? If so i'll create a new mailing list pypy-issue or so and have things rather sent there. Just let me know. Personally i find it a bit unfortuante that the many closed/resolved issues are not notified to this list as well :) cheers, holger From issues-noreply at bitbucket.org Sat Sep 4 14:57:54 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Sat, 04 Sep 2010 12:57:54 -0000 Subject: [py-dev] New issue 117 in py-trunk: setuptools errors bubble up in a unhelpfull way Message-ID: <4351b6401465d35da867a99981df9ade@bitbucket.org> --- you can reply above this line --- New issue 117: setuptools errors bubble up in a unhelpfull way http://bitbucket.org/hpk42/py-trunk/issue/117/setuptools-errors-bubble-up-in-a-unhelpfull Ronny Pfannschmidt / RonnyPfannschmidt on Sat, 4 Sep 2010 14:57:54 +0200: Description: i get the following trace because the setuptools errors just bubble up what i really needed to know is the EntryPoint as well, cause the origin was the still installed pytest_cov plugin {{{#!pytrace $ py.test inserting into sys.path: /home/ronny/Projects/py Traceback (most recent call last): File "/home/ronny/Projects/py/bin/py.test", line 3, in py.cmdline.pytest() File "/home/ronny/Projects/py/py/_test/cmdline.py", line 13, in main config.parse(args) File "/home/ronny/Projects/py/py/_test/config.py", line 95, in parse self._preparse(args) File "/home/ronny/Projects/py/py/_test/config.py", line 83, in _preparse self.pluginmanager.consider_setuptools_entrypoints() File "/home/ronny/Projects/py/py/_test/pluginmanager.py", line 104, in consider_setuptools_entrypoints plugin = ep.load() File "/home/ronny/Projects/distribute/pkg_resources.py", line 1953, in load if require: self.require(env, installer) File "/home/ronny/Projects/distribute/pkg_resources.py", line 1966, in require working_set.resolve(self.dist.requires(self.extras),env,installer)) File "/home/ronny/Projects/distribute/pkg_resources.py", line 552, in resolve raise DistributionNotFound(req) pkg_resources.DistributionNotFound: pytest-xdist>=1.4 }}} Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From flub at devork.be Sun Sep 5 23:42:19 2010 From: flub at devork.be (Floris Bruynooghe) Date: Sun, 5 Sep 2010 22:42:19 +0100 Subject: [py-dev] Advanced assert equal In-Reply-To: <20100902092722.GG1914@trillke.net> References: <20100816002557.GA12335@laurie.devork.be> <1281958153.12940.5.camel@klappe2> <20100816135154.GA24594@laurie.devork.be> <20100902092722.GG1914@trillke.net> Message-ID: <20100905214219.GA31056@laurie.devork.be> Hello Holger On Thu, Sep 02, 2010 at 11:27:22AM +0200, holger krekel wrote: > would have the AST-transformation call back into a generic > pytest-hook if an assertion fails. So one could customize > "hasattr(x, y)" or "x in y" or "x == y == z" etc. and all > such representation code would move to the py/_plugin/pytest_assertion.py > plugin. That being said going for just comparisons right now > is fine, let's not have more general thoughts hold us up. I wouldn't want to make this more difficult or incompatible though, so I'd be interested in how you where thinking of doing this. I guess this would be exposing the various .visit_*() methods in _assertionnew.py in some way? If so I think there's also value in the more simple pytest_assert_compare(op, left, right) I'm proposing here (and others like "in", "is" etc) as they don't require having any AST knowledge to implement, at a cost of slightly less flexibility. > On Mon, Aug 16, 2010 at 14:51 +0100, Floris Bruynooghe wrote: > > A possible api for the hook > > could be: > > > > def pytest_assert_compare(op, left, right, left_repr, right_repr): > > """Customise compare > > > > Return None for no custom compare, otherwise return a list of > > strings. The strings will be joined by newlines but any newlines > > *in* a string will be escaped. > > """ > > pass > > > > I guess the reprs are not really necessary if there's an easy way to > > make them. It's just that I used them in my original patch. > > Hum, couldn't custom representations just mean that there is no > "history" of where the object comes from? > > The hook looks otherwise good to me. I've made this hook pytest_assert_compare(op, left, right) now, as I've found py.io.saferepr() which seems to do a good job. Not sure what you're referring too with the "history" comment. If you mean that as soon as a specialised hook is found the previous explanations (e.g. from Assert AST node) get lost I'd like to disagree. It seems reasonable for the new hooks to just provide a more detailed/specialised explanation of a particular part of the failure rather then replace the entire explanation. Otherwise you might also lose detail. > (I guess you are aware that any pytest-hook implementation can always > choose to accept less than the available arguments). I wasn't actually, neat. > > There's also the question of who should truncate large amounts of data > > (e.g. screenfulls of diffs): the hook itself, the caller of the hook > > or _format_explanation()? Probably one of the first two to get rid of > > the memory usage as soon as possible. > > If a hook returns something we should (probably) not further do anything > with it in _format_explanation(). And making truncation the repsonsibility > of the hook makes sense i think. ok > I am not sure about the general use cases, from my side: > > x == y > x != y > x in y > > are the interesting ones (with some of the objects being long > lists, long strings etc.). so some hook for a "binary" relation > makes sense, pytest_assert_binrepr(op) where op could be "==", > "in" or "is" etc. To me it seems more logical to add a separate hook for each .visit_*() method rather then put multiple together. But maybe that seems artificial from a user point of view? > Floris, i'd very much like to start using/having improved assertion > representations. Happy to review patches or forks for inclusion. I've attached a new patch in which I attempt to use the hook system and added a pytest_assert_comare(op, left, right) hook. I must admit I don't fully understand the plugin/hook system so hope I did it right [0]. Again I've not concentrated on the actual specific comparisons, rather would like to get a reasonable idea of how good the general approach is. If you like this version I can create a fork on bitbucket and start working on more/better hook implementations. Regards Floris [0] I assume that py.test.config.hook.pytest_assert_compare is the function to call and that it returns a list with the results of each such function found, with the first element being the most "specific" result. But I just figured that out using trial and error rather then understand the plugin system. -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org -------------- next part -------------- A non-text attachment was scrubbed... Name: pytest_assert.diff Type: text/x-diff Size: 8410 bytes Desc: not available URL: From holger at merlinux.eu Mon Sep 6 10:33:53 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 6 Sep 2010 10:33:53 +0200 Subject: [py-dev] Advanced assert equal In-Reply-To: <20100905214219.GA31056@laurie.devork.be> References: <20100816002557.GA12335@laurie.devork.be> <1281958153.12940.5.camel@klappe2> <20100816135154.GA24594@laurie.devork.be> <20100902092722.GG1914@trillke.net> <20100905214219.GA31056@laurie.devork.be> Message-ID: <20100906083353.GP32478@trillke.net> Hi Floris, On Sun, Sep 05, 2010 at 22:42 +0100, Floris Bruynooghe wrote: > Hello Holger > > On Thu, Sep 02, 2010 at 11:27:22AM +0200, holger krekel wrote: > > would have the AST-transformation call back into a generic > > pytest-hook if an assertion fails. So one could customize > > "hasattr(x, y)" or "x in y" or "x == y == z" etc. and all > > such representation code would move to the py/_plugin/pytest_assertion.py > > plugin. That being said going for just comparisons right now > > is fine, let's not have more general thoughts hold us up. > > I wouldn't want to make this more difficult or incompatible though, so > I'd be interested in how you where thinking of doing this. I guess > this would be exposing the various .visit_*() methods in > _assertionnew.py in some way? If so I think there's also value in the > more simple pytest_assert_compare(op, left, right) I'm proposing here > (and others like "in", "is" etc) as they don't require having any AST > knowledge to implement, at a cost of slightly less flexibility. Sure. The eventual callback that is to receive AST-nodes can remain internal and there is nothing from keeping us to offer both levels of hooks (the default impl for the low-level hook can just call the higher level ones). > > On Mon, Aug 16, 2010 at 14:51 +0100, Floris Bruynooghe wrote: > > > A possible api for the hook > > > could be: > > > > > > def pytest_assert_compare(op, left, right, left_repr, right_repr): > > > """Customise compare > > > > > > Return None for no custom compare, otherwise return a list of > > > strings. The strings will be joined by newlines but any newlines > > > *in* a string will be escaped. > > > """ > > > pass > > > > > > I guess the reprs are not really necessary if there's an easy way to > > > make them. It's just that I used them in my original patch. > > > > Hum, couldn't custom representations just mean that there is no > > "history" of where the object comes from? > > > > The hook looks otherwise good to me. > > I've made this hook pytest_assert_compare(op, left, right) now, as > I've found py.io.saferepr() which seems to do a good job. ok. > Not sure what you're referring too with the "history" comment. If you > mean that as soon as a specialised hook is found the previous > explanations (e.g. from Assert AST node) get lost I'd like to > disagree. It seems reasonable for the new hooks to just provide a > more detailed/specialised explanation of a particular part of the > failure rather then replace the entire explanation. Otherwise you > might also lose detail. Hum, ok. Let's see how this plays out with real failures. > > (I guess you are aware that any pytest-hook implementation can always > > choose to accept less than the available arguments). > > I wasn't actually, neat. > > > > There's also the question of who should truncate large amounts of data > > > (e.g. screenfulls of diffs): the hook itself, the caller of the hook > > > or _format_explanation()? Probably one of the first two to get rid of > > > the memory usage as soon as possible. > > > > If a hook returns something we should (probably) not further do anything > > with it in _format_explanation(). And making truncation the repsonsibility > > of the hook makes sense i think. > > ok > > > I am not sure about the general use cases, from my side: > > > > x == y > > x != y > > x in y > > > > are the interesting ones (with some of the objects being long > > lists, long strings etc.). so some hook for a "binary" relation > > makes sense, pytest_assert_binrepr(op) where op could be "==", > > "in" or "is" etc. > > To me it seems more logical to add a separate hook for each .visit_*() > method rather then put multiple together. But maybe that seems > artificial from a user point of view? I guess I prefer "pytest_assert_binop(op, val1, val2)" but keeping strictly to the visit relation also has merits. I guess my preference also comes from trying to avoid introducing too many hooks. > > Floris, i'd very much like to start using/having improved assertion > > representations. Happy to review patches or forks for inclusion. > > I've attached a new patch in which I attempt to use the hook system > and added a pytest_assert_comare(op, left, right) hook. I must admit > I don't fully understand the plugin/hook system so hope I did it > right [0]. Again I've not concentrated on the actual specific > comparisons, rather would like to get a reasonable idea of how good > the general approach is. Your hook implementation looks fine and [0] is correct. Please feel free to ask any questions regarding hook machinery here on the list. Are you aware of (the rather minimal) documentation here? http://codespeak.net/py/dist/test/customize.html#important-py-test-hooks > If you like this version I can create a fork on bitbucket and start > working on more/better hook implementations. Great, please do so. Two review notes regarding your patch already: * please try to avoid global imports in plugins, rather only import e.g. difflib/pprint where you actually use it. This is to try to keep interactive startup-time minimal, particularly wrt builtin plugins. * i suggest to put tests to testing/plugin/test_pytest_assertion.py (rather than testing/code/...) especially since the tested functionality is contained in the pytest_assertion.py plugin. For direct unit tests you may also directly do "from py._plugin.pytest_assertion import XYZ". best, holger > Regards > Floris > > > [0] I assume that py.test.config.hook.pytest_assert_compare is the > function to call and that it returns a list with the results of > each such function found, with the first element being the most > "specific" result. But I just figured that out using trial and > error rather then understand the plugin system. > > -- > Debian GNU/Linux -- The Power of Freedom > www.debian.org | www.gnu.org | www.kernel.org > diff --git a/py/_code/_assertionnew.py b/py/_code/_assertionnew.py > --- a/py/_code/_assertionnew.py > +++ b/py/_code/_assertionnew.py > @@ -162,10 +162,7 @@ class DebugInterpreter(ast.NodeVisitor): > def visit_Compare(self, comp): > left = comp.left > left_explanation, left_result = self.visit(left) > - got_result = False > for op, next_op in zip(comp.ops, comp.comparators): > - if got_result and not result: > - break > next_explanation, next_result = self.visit(next_op) > op_symbol = operator_map[op.__class__] > explanation = "%s %s %s" % (left_explanation, op_symbol, > @@ -177,9 +174,16 @@ class DebugInterpreter(ast.NodeVisitor): > __exprinfo_right=next_result) > except Exception: > raise Failure(explanation) > - else: > - got_result = True > + if not result: > + break > left_explanation, left_result = next_explanation, next_result > + hook_result = py.test.config.hook.pytest_assert_compare( > + op=op_symbol, left=left_result, right=next_result) > + if hook_result: > + for new_expl in hook_result: > + if result: > + explanation = '\n~'.join(new_expl) > + break > return explanation, result > > def visit_BoolOp(self, boolop): > diff --git a/py/_code/assertion.py b/py/_code/assertion.py > --- a/py/_code/assertion.py > +++ b/py/_code/assertion.py > @@ -5,12 +5,20 @@ BuiltinAssertionError = py.builtin.built > > > def _format_explanation(explanation): > - # uck! See CallFunc for where \n{ and \n} escape sequences are used > + """This formats an explanation > + > + Normally all embedded newlines are escaped, however there are > + three exceptions: \n{, \n} and \n~. The first two are intended > + cover nested explanations, see function and attribute explanations > + for examples (.visit_Call(), visit_Attribute()). The last one is > + for when one explanation needs to span multiple lines, e.g. when > + displaying diffs. > + """ > raw_lines = (explanation or '').split('\n') > - # escape newlines not followed by { and } > + # escape newlines not followed by {, } and ~ > lines = [raw_lines[0]] > for l in raw_lines[1:]: > - if l.startswith('{') or l.startswith('}'): > + if l.startswith('{') or l.startswith('}') or l.startswith('~'): > lines.append(l) > else: > lines[-1] += '\\n' + l > @@ -28,11 +36,14 @@ def _format_explanation(explanation): > stackcnt[-1] += 1 > stackcnt.append(0) > result.append(' +' + ' '*(len(stack)-1) + s + line[1:]) > - else: > + elif line.startswith('}'): > assert line.startswith('}') > stack.pop() > stackcnt.pop() > result[stack[-1]] += line[1:] > + else: > + assert line.startswith('~') > + result.append(' '*len(stack) + line[1:]) > assert len(stack) == 1 > return '\n'.join(result) > > diff --git a/py/_plugin/hookspec.py b/py/_plugin/hookspec.py > --- a/py/_plugin/hookspec.py > +++ b/py/_plugin/hookspec.py > @@ -124,6 +124,19 @@ def pytest_sessionfinish(session, exitst > """ whole test run finishes. """ > > # ------------------------------------------------------------------------- > +# hooks for customising the assert methods > +# ------------------------------------------------------------------------- > + > +def pytest_assert_compare(op, left, right): > + """Customise compare assertion > + > + Return None or an empty list for no custom compare, otherwise > + return a list of strings. The strings will be joined by newlines > + but any newlines *in* as string will be escaped. Note that all > + but the first line will be indented sligthly. > + """ > + > +# ------------------------------------------------------------------------- > # hooks for influencing reporting (invoked from pytest_terminal) > # ------------------------------------------------------------------------- > > diff --git a/py/_plugin/pytest_assertion.py b/py/_plugin/pytest_assertion.py > --- a/py/_plugin/pytest_assertion.py > +++ b/py/_plugin/pytest_assertion.py > @@ -1,3 +1,6 @@ > +import difflib > +import pprint > + > import py > import sys > > @@ -26,3 +29,49 @@ def warn_about_missing_assertion(): > else: > py.std.warnings.warn("Assertions are turned off!" > " (are you using python -O?)") > + > + > +def pytest_assert_compare(op, left, right): > + """Make a specialised explanation for comapare equal""" > + if op != '==' or type(left) != type(right): > + return None > + explanation = [] > + left_repr = py.io.saferepr(left, maxsize=30) > + right_repr = py.io.saferepr(right, maxsize=30) > + explanation += ['%s == %s' % (left_repr, right_repr)] > + issquence = lambda x: isinstance(x, (list, tuple)) > + istext = lambda x: isinstance(x, basestring) > + isdict = lambda x: isinstance(x, dict) > + if istext(left): > + explanation += [line.strip('\n') for line in > + difflib.ndiff(left.splitlines(), right.splitlines())] > + elif issquence(left): > + explanation += _compare_eq_sequence(left, right) > + elif isdict(left): > + explanation += _pprint_diff(left, right) > + else: > + return None # No specialised knowledge > + return explanation > + > + > +def _compare_eq_sequence(left, right): > + explanation = [] > + for i in xrange(min(len(left), len(right))): > + if left[i] != right[i]: > + explanation += ['First differing item %s: %s != %s' % > + (i, left[i], right[i])] > + break > + if len(left) > len(right): > + explanation += ['Left contains more items, ' > + 'first extra item: %s' % left[len(right)]] > + elif len(left) < len(right): > + explanation += ['Right contains more items, ' > + 'first extra item: %s' % right[len(right)]] > + return explanation + _pprint_diff(left, right) > + > + > +def _pprint_diff(left, right): > + """Make explanation using pprint and difflib""" > + return [line.strip('\n') for line in > + difflib.ndiff(pprint.pformat(left).splitlines(), > + pprint.pformat(right).splitlines())] > diff --git a/testing/code/test_assertionnew.py b/testing/code/test_assertionnew.py > new file mode 100644 > --- /dev/null > +++ b/testing/code/test_assertionnew.py > @@ -0,0 +1,74 @@ > +import sys > + > +import py > +from py._code._assertionnew import interpret > + > + > +def getframe(): > + """Return the frame of the caller as a py.code.Frame object""" > + return py.code.Frame(sys._getframe(1)) > + > + > +def setup_module(mod): > + py.code.patch_builtins(assertion=True, compile=False) > + > + > +def teardown_module(mod): > + py.code.unpatch_builtins(assertion=True, compile=False) > + > + > +def test_assert_simple(): > + # Simply test that this way of testing works > + a = 0 > + b = 1 > + r = interpret('assert a == b', getframe()) > + assert r == 'assert 0 == 1' > + > + > +def test_assert_list(): > + r = interpret('assert [0, 1] == [0, 2]', getframe()) > + msg = ('assert [0, 1] == [0, 2]\n' > + ' First differing item 1: 1 != 2\n' > + ' - [0, 1]\n' > + ' ? ^\n' > + ' + [0, 2]\n' > + ' ? ^') > + print r > + assert r == msg > + > + > +def test_assert_string(): > + r = interpret('assert "foo and bar" == "foo or bar"', getframe()) > + msg = ("assert 'foo and bar' == 'foo or bar'\n" > + " - foo and bar\n" > + " ? ^^^\n" > + " + foo or bar\n" > + " ? ^^") > + print r > + assert r == msg > + > + > +def test_assert_multiline_string(): > + a = 'foo\nand bar\nbaz' > + b = 'foo\nor bar\nbaz' > + r = interpret('assert a == b', getframe()) > + msg = ("assert 'foo\\nand bar\\nbaz' == 'foo\\nor bar\\nbaz'\n" > + ' foo\n' > + ' - and bar\n' > + ' + or bar\n' > + ' baz') > + print r > + assert r == msg > + > + > +def test_assert_dict(): > + a = {'a': 0, 'b': 1} > + b = {'a': 0, 'c': 2} > + r = interpret('assert a == b', getframe()) > + msg = ("assert {'a': 0, 'b': 1} == {'a': 0, 'c': 2}\n" > + " - {'a': 0, 'b': 1}\n" > + " ? ^ ^\n" > + " + {'a': 0, 'c': 2}\n" > + " ? ^ ^") > + print r > + assert r == msg > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev -- From issues-noreply at bitbucket.org Tue Sep 7 17:02:27 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Tue, 07 Sep 2010 15:02:27 -0000 Subject: [py-dev] New issue 118 in py-trunk: native traceback option Message-ID: --- you can reply above this line --- New issue 118: native traceback option http://bitbucket.org/hpk42/py-trunk/issue/118/native-traceback-option nikow on Tue, 7 Sep 2010 17:02:27 +0200: Description: I'm using Eclipse with PyDev and when running unittest this allows me to click on the traceback information to directly jump to the code. For py.test when called from the IDE this does not work, since apparently PyDev relies on the standard traceback format to make it clickable. Therefore it would be great if a --tb option was added to display tracebacks in the native formatting. For now I fixed this by inserting the following code in collect.py in line 177: {{{ import traceback tb_strs = traceback.format_tb(excinfo._excinfo[2]) i = 0 while not tb_strs[i].endswith(" testfunction(**funcargs)\n"): i += 1 tb_strs = tb_strs[i+1:] return "\n" + "".join(tb_strs) + "\n" }}} On a side note: I'm currently use {{{py.test.cmdline.main(...)}}} to run py.test from inside the IDE. Maybe one could add some "offical" API to call py.test in this way. Thanks. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From flub at devork.be Wed Sep 8 23:38:02 2010 From: flub at devork.be (Floris Bruynooghe) Date: Wed, 8 Sep 2010 22:38:02 +0100 Subject: [py-dev] Advanced assert equal In-Reply-To: <20100906083353.GP32478@trillke.net> References: <20100816002557.GA12335@laurie.devork.be> <1281958153.12940.5.camel@klappe2> <20100816135154.GA24594@laurie.devork.be> <20100902092722.GG1914@trillke.net> <20100905214219.GA31056@laurie.devork.be> <20100906083353.GP32478@trillke.net> Message-ID: <20100908213802.GA27981@laurie.devork.be> Hello On Mon, Sep 06, 2010 at 10:33:53AM +0200, holger krekel wrote: > On Sun, Sep 05, 2010 at 22:42 +0100, Floris Bruynooghe wrote: > > On Thu, Sep 02, 2010 at 11:27:22AM +0200, holger krekel wrote: > > > On Mon, Aug 16, 2010 at 14:51 +0100, Floris Bruynooghe wrote: > > > Hum, couldn't custom representations just mean that there is no > > > "history" of where the object comes from? [...] > > Not sure what you're referring too with the "history" comment. If you > > mean that as soon as a specialised hook is found the previous > > explanations (e.g. from Assert AST node) get lost I'd like to > > disagree. It seems reasonable for the new hooks to just provide a > > more detailed/specialised explanation of a particular part of the > > failure rather then replace the entire explanation. Otherwise you > > might also lose detail. > > Hum, ok. Let's see how this plays out with real failures. Agreed, I don't feel too strongly about it but would like to see how it works out as it is now. > > To me it seems more logical to add a separate hook for each .visit_*() > > method rather then put multiple together. But maybe that seems > > artificial from a user point of view? > > I guess I prefer "pytest_assert_binop(op, val1, val2)" > but keeping strictly to the visit relation also has merits. > I guess my preference also comes from trying to avoid introducing > too many hooks. Again I don't feel too strongly, you've got more say on the hook API then me so if you want I can rename it and we can see how it looks once there are more implementations for e.g. "in", "is" etc. > > If you like this version I can create a fork on bitbucket and start > > working on more/better hook implementations. > > Great, please do so. Two review notes regarding your patch already: > > * please try to avoid global imports in plugins, rather only import > e.g. difflib/pprint where you actually use it. This is to try to > keep interactive startup-time minimal, particularly wrt builtin plugins. > > * i suggest to put tests to testing/plugin/test_pytest_assertion.py > (rather than testing/code/...) especially since the tested functionality > is contained in the pytest_assertion.py plugin. For direct unit tests > you may also directly do "from py._plugin.pytest_assertion import XYZ". As you've probably noticed I've made a fork at http://bitbucket.org/flub/py-trunk-assert. I've addressed these two points in it already, or so I hope. I can't figure out how to properly test the plugin however, it either feels too brittle or too loose. So any hints on how to test that sort of thing would be welcome. Other then that I need to get on with implementing more and better default behaviour, at least cover everything unittest2 does. Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From holger at merlinux.eu Thu Sep 9 11:07:44 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 9 Sep 2010 11:07:44 +0200 Subject: [py-dev] Advanced assert equal In-Reply-To: <20100908213802.GA27981@laurie.devork.be> References: <20100816002557.GA12335@laurie.devork.be> <1281958153.12940.5.camel@klappe2> <20100816135154.GA24594@laurie.devork.be> <20100902092722.GG1914@trillke.net> <20100905214219.GA31056@laurie.devork.be> <20100906083353.GP32478@trillke.net> <20100908213802.GA27981@laurie.devork.be> Message-ID: <20100909090744.GX32478@trillke.net> On Wed, Sep 08, 2010 at 22:38 +0100, Floris Bruynooghe wrote: > As you've probably noticed I've made a fork at > http://bitbucket.org/flub/py-trunk-assert. I've addressed these two > points in it already, or so I hope. I can't figure out how to > properly test the plugin however, it either feels too brittle or too > loose. So any hints on how to test that sort of thing would be > welcome. The tests you added in http://bitbucket.org/flub/py-trunk-assert/src/c8e85f1f4adb/testing/plugin/test_pytest_assertion.py look fine to me. You could check for a bit more detail on some selected examples but the main thing is to make sure that nothing blows up. Regarding the tests in http://bitbucket.org/flub/py-trunk-assert/src/c8e85f1f4adb/testing/code/test_assertionnew.py i'd rather move them to the above plugin test module. py/code is completely independent from py.test. Speaking of it, in _assertionew.py your very call to py.test.config.pytest_assert_compare() is problematic. ``py.test.config`` is a global access path that is never used internally. Not too far off i'd like to get rid of ``py.test.config`` alltogether or rather move it to a "pytest_compat" plugin which maintains some older APIs/behaviours. For your branch, it means that the pytest_assertion plugin needs to parametrize the assertion-machinery with a config/hook object. Maybe for now, we can just do something like: def pytest_configure(config): ... py.builtin.builtins.AssertionError._pytesthook = config.hook and check for and call through this attribute from _assertionnew.py. This means that if one uses py/code without py.test then the latter will be not be initialized/touched at all (accessing py.test.config triggers default plugin loading and basic initialization). > Other then that I need to get on with implementing more and better > default behaviour, at least cover everything unittest2 does. Great, thanks. This looks all quite good. Looking forward to use it myself soon :) best, holger From issues-noreply at bitbucket.org Tue Sep 14 13:49:29 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Tue, 14 Sep 2010 11:49:29 -0000 Subject: [py-dev] New issue 119 in py-trunk: Fail to impot __init__.py - INTERNALERROR> self = Message-ID: --- you can reply above this line --- New issue 119: Fail to impot __init__.py - INTERNALERROR> self = http://bitbucket.org/hpk42/py-trunk/issue/119/fail-to-impot-__init__py-internalerror sorin on Tue, 14 Sep 2010 13:49:29 +0200: Description: For some reason py.test fails on any __init__.py file. How to reproduce: * create an empty __init__.py * execute py.test --collectonly INTERNALERROR> self = INTERNALERROR> colitems = [] INTERNALERROR> INTERNALERROR> def main(self, colitems): INTERNALERROR> """ main loop for running tests. """ INTERNALERROR> self.shouldstop = False INTERNALERROR> self.sessionstarts() INTERNALERROR> exitstatus = EXIT_OK INTERNALERROR> try: INTERNALERROR> > self._mainloop(colitems) INTERNALERROR> INTERNALERROR> colitems = [] INTERNALERROR> exitstatus = 0 INTERNALERROR> self = INTERNALERROR> INTERNALERROR> C:\lib\Python26.x86\lib\site-packages\py-1.3.3-py2.6.egg\py\_test\session.py:113: INTERNALERROR> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ... Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Tue Sep 14 15:20:15 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Tue, 14 Sep 2010 13:20:15 -0000 Subject: [py-dev] New issue 120 in py-trunk: Add collect_include option to complement collect_ignore option. Message-ID: --- you can reply above this line --- New issue 120: Add collect_include option to complement collect_ignore option. http://bitbucket.org/hpk42/py-trunk/issue/120/add-collect_include-option-to-complement-collect_ignore sorin on Tue, 14 Sep 2010 15:20:15 +0200: Description: On big/huge project people may want to scan only some directories, and in order to obtain this the current `collect_ignore` list is not enough. Adding tons of directories to `collect_ignore` may not be the best practice instead of just adding the locations you want to be scanned. I would like to propose the addition of `collect_include` option that would allow only the files matching the patterns from it to be scanned. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From holger at merlinux.eu Tue Sep 14 17:34:33 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 14 Sep 2010 17:34:33 +0200 Subject: [py-dev] pylib/py.test 1.3.4: fixes and new native traceback option Message-ID: <20100914153433.GM32478@trillke.net> pylib/py.test 1.3.4 is a minor maintenance release mostly containing bug fixes and a new "--tb=native" traceback option to show "normal" Python standard tracebacks instead of the py.test enhanced tracebacks. See below for more change info and http://pytest.org for more general information on features and configuration of the testing tool. Thanks to the issue reporters and generally to Ronny Pfannschmidt for help. cheers, holger krekel Changes between 1.3.3 and 1.3.4 ================================================== - fix issue111: improve install documentation for windows - fix issue119: fix custom collectability of __init__.py as a module - fix issue116: --doctestmodules work with __init__.py files as well - fix issue115: unify internal exception passthrough/catching/GeneratorExit - fix issue118: new --tb=native for presenting cpython-standard exceptions -- From flub at devork.be Wed Sep 22 02:11:27 2010 From: flub at devork.be (Floris Bruynooghe) Date: Wed, 22 Sep 2010 01:11:27 +0100 Subject: [py-dev] Advanced assert equal In-Reply-To: <20100909090744.GX32478@trillke.net> References: <20100816002557.GA12335@laurie.devork.be> <1281958153.12940.5.camel@klappe2> <20100816135154.GA24594@laurie.devork.be> <20100902092722.GG1914@trillke.net> <20100905214219.GA31056@laurie.devork.be> <20100906083353.GP32478@trillke.net> <20100908213802.GA27981@laurie.devork.be> <20100909090744.GX32478@trillke.net> Message-ID: Hi Holger I've updated my repo with the following changes: * Don't use py.test.config, using Holger's suggestion of an attribute on AssertionError * Renamed pytest_assert_compare to pytest_assert_binrepr * Moved all tests to testing/plugin/test_pytest_assertion.py * Implemented the default hook to pretty much mirror unittest2's behaviour. * Added examples in doc/examples/assertion/failure_demo.py - I found this easier to work with then the actual tests I guess the main thing now is to fine-tune the default reporting for various cases. I'm fairly tempted not to try and predict too much in this area and rather wait to tune these till they come up. Related to this you said you'd like to see "x in y" and "x != y" for some types. Could you be more specific as I can't think of what you'd like to do with "in" or "!=", I'm probably lacking creativity right now... I'm happy to hear feedback anyone might have, I've probably still missed things. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From holger at merlinux.eu Wed Sep 22 11:55:54 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 22 Sep 2010 11:55:54 +0200 Subject: [py-dev] Advanced assert equal In-Reply-To: References: <20100816002557.GA12335@laurie.devork.be> <1281958153.12940.5.camel@klappe2> <20100816135154.GA24594@laurie.devork.be> <20100902092722.GG1914@trillke.net> <20100905214219.GA31056@laurie.devork.be> <20100906083353.GP32478@trillke.net> <20100908213802.GA27981@laurie.devork.be> <20100909090744.GX32478@trillke.net> Message-ID: <20100922095554.GH15851@trillke.net> hi Floris, On Wed, Sep 22, 2010 at 01:11 +0100, Floris Bruynooghe wrote: > Hi Holger > > I've updated my repo with the following changes: > > * Don't use py.test.config, using Holger's suggestion of an attribute > on AssertionError > * Renamed pytest_assert_compare to pytest_assert_binrepr > * Moved all tests to testing/plugin/test_pytest_assertion.py > * Implemented the default hook to pretty much mirror unittest2's behaviour. > * Added examples in doc/examples/assertion/failure_demo.py - I found > this easier to work with then the actual tests > > > I guess the main thing now is to fine-tune the default reporting for > various cases. I'm fairly tempted not to try and predict too much in > this area and rather wait to tune these till they come up. Yes, this makes sense. I am eager to merge, a few things: * could you merge current py-trunk? * could you run tests on python3? * could you maybe look into the two failures i attached as test_bugs.py? (feel free to hang out/ask on #pylib - i should be there a lot this week and today). > Related to this you said you'd like to see "x in y" and "x != y" for > some types. Could you be more specific as I can't think of what you'd > like to do with "in" or "!=", I'm probably lacking creativity right > now... Let's see when i or someone else actually comes up with a concrete case :) > I'm happy to hear feedback anyone might have, I've probably still missed things. Thanks already for your work, great additions. best, holger > > Regards > Floris > > -- > Debian GNU/Linux -- The Power of Freedom > www.debian.org | www.gnu.org | www.kernel.org > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- -------------- next part -------------- A non-text attachment was scrubbed... Name: test_bugs.py Type: text/x-python Size: 235 bytes Desc: not available URL: From issues-noreply at bitbucket.org Mon Sep 27 11:35:10 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Mon, 27 Sep 2010 09:35:10 -0000 Subject: [py-dev] New issue 121 in py-trunk: py.test seems gather tests from upper directories Message-ID: <8ae1cb022651e3d466d763c5550d3537@bitbucket.org> --- you can reply above this line --- New issue 121: py.test seems gather tests from upper directories http://bitbucket.org/hpk42/py-trunk/issue/121/pytest-seems-gather-tests-from-upper bluebird75 on Mon, 27 Sep 2010 11:35:10 +0200: Description: I don't know what's wrong with my setup, but since I upgraded py.test , it no longer restrict itself to the current directory and subdirectories, it looks up for other directories. {{{ #!bash hilippe at pc-philippe /cygdrive/d/work/elc-dev $ ls -d elc* elc-memory/ elc-py26/ elc-work-fh1/ elc2/ elc-merge-linkchecker/ elc-ref/ elc1/ elc3/ Philippe at pc-philippe /cygdrive/d/work/elc-dev $ cd elc1 Philippe at pc-philippe /cygdrive/d/work/elc-dev/elc1 $ py.test 2010-09-27 11:30:59,592 5404 INFO elc.ElcLogger init_logging_local done C:\Python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_plugin\pytest_default.py:57: DeprecationWarning: was found in a conftest.py file, use pytest_collect hooks instead. (since version >1.1) Directory = parent.config._getcollectclass('Directory', path) Traceback (most recent call last): File "c:\Python26\Scripts\py.test-script.py", line 8, in load_entry_point('py==1.3.4', 'console_scripts', 'py.test')() File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_cmdline\pytest.py", line 5, in main raise SystemExit(py.test.cmdline.main(args)) File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\cmdline.py", line 16, in main colitems = config.getinitialnodes() File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\config.py", line 158, in getinitialnodes return [self.getnode(arg) for arg in self.args] File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\config.py", line 173, in getnode return self._rootcol.getbynames(names) File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\collect.py", line 388, in getbynames for x in current._memocollect(): File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\collect.py", line 230, in _memocollect return self._memoizedcall('_collected', self.collect) File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\collect.py", line 104, in _memoizedcall res = function() File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\collect.py", line 300, in collect res = self.consider(path) File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\collect.py", line 309, in consider if self.ihook.pytest_ignore_collect(path=path, config=self.config): File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\collect.py", line 21, in call_matching_hooks return hookmethod.pcall(plugins, **kwargs) File "C:\Python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\pluginmanager.py", line 354, in pcall return self.hookrelay._performcall(self.name, mc) File "C:\Python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\pluginmanager.py", line 335, in _performcall return multicall.execute() File "C:\Python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\pluginmanager.py", line 244, in execute res = method(**kwargs) File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_plugin\pytest_default.py", line 32, in pytest_ignore_collect ignore_paths = config.getconftest_pathlist("collect_ignore", path=path) File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\config.py", line 195, in getconftest_pathlist mod, relroots = self._conftest.rget_with_confmod(name, path) File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\conftesthandle.py", line 88, in rget_with_confmod modules = self.getconftestmodules(path) File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\conftesthandle.py", line 72, in getconftestmodules clist.append(self.importconftest(conftestpath)) File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_test\conftesthandle.py", line 108, in importconftest mod = conftestpath.pyimport() File "c:\python26\lib\site-packages\py-1.3.4-py2.6.egg\py\_path\local.py", line 540, in pyimport raise self.ImportMismatchError(modname, modfile, self) py._path.local.ImportMismatchError: ('conftest', 'd:\\work\\elc-dev\\Copy of git-linkchecker\\conftest.py', local('d:\\work\\elc-dev\\elc-memory\\conftest.py')) Philippe at pc-philippe /cygdrive/d/work/elc-dev/elc1 $ }}} The errors reported below belong to other directories than elc1. I have a conftest.py in elc1: {{{ #!python Philippe at pc-philippe /cygdrive/d/work/elc-dev/elc1 $ cat conftest.py # this module is loaded by py.test before running the tests import fix_path import elcloginit import tests.httpd_utils as httpd_utils from tests.setupDbTestEnv import SetupDbTestEnv, envDict collect_ignore = [ 'elc_linkchecker/linkchecker/' ] DB_TEST_ENV = None def pytest_namespace(): global DB_TEST_ENV DB_TEST_ENV = SetupDbTestEnv() envDict[ 'dbTestEnv' ] = DB_TEST_ENV return envDict def pytest_sessionstart( session ): # print 'pytest_sessionstart' DB_TEST_ENV.setup() def pytest_sessionfinish( session, exitstatus ): # print 'pytest_sessionfinish' DB_TEST_ENV.teardown() pass }}} Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Tue Sep 28 16:33:23 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Tue, 28 Sep 2010 14:33:23 -0000 Subject: [py-dev] New issue 122 in py-trunk: store/reuse the state of a last run from a previous session dir Message-ID: <6b06d9807e28da196730f1d928f3c92f@bitbucket.org> --- you can reply above this line --- New issue 122: store/reuse the state of a last run from a previous session dir http://bitbucket.org/hpk42/py-trunk/issue/122/store-reuse-the-state-of-a-last-run-from-a-previous-session Ronny Pfannschmidt / RonnyPfannschmidt on Tue, 28 Sep 2010 16:33:23 +0200: Description: taking failures/not yet executed items from the last session dir could help in various cases to prevent tests from rerunning Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From flub at devork.be Wed Sep 29 01:12:53 2010 From: flub at devork.be (Floris Bruynooghe) Date: Wed, 29 Sep 2010 00:12:53 +0100 Subject: [py-dev] controlling assertions and difflib Message-ID: Hi As Holger pointed out the following test hangs py.test in my assertion clone, it triggers a bug in difflib: def test_hang_in_diff(): x = "1\n" * 1000 + "abc" + "2\n" * 1000 y = "1\n" * 1000 + "def" + "2\n" * 1000 assert x == y It seems difflib can not deal with the 1000 identical trailing items. It fails with a TypeError somewhere very deep in a recursion or, when consuming the generator inside a list comprehension, just completely hangs. Also before failing it considers all the 2s as different too. Rather then debugging difflib (there are several possible bugs on bugs.python.org which might relate to this) I've come up with some code that does produce this instead: ______________________________ test_hang_in_diff _______________________________ /tmp/sandbox/test_bugs.py:4: in test_hang_in_diff > assert x == y E assert '1\n1\n1\n1\n...n2\n2\n2\n2\n' == '1\n1\n1\n1\n...n2\n2\n2\n2\n' E Skipping 1990 identical leading characters in diff E Skipping 1991 identical trailing characters in diff E 1 E 1 E 1 E 1 E 1 E - abc2 E + def2 E 2 E 2 E 2 E 2 Another solution is to use context_diff() which does not have this problem, but for some reason also considers all the trailing 2s to be different. Additionally context diff also doesn't provide the nice markers for differences inside a line which might be problematic as there's no guarantee there will be nice line endings in the input. In general I was already thinking of cutting off similar prefixes and suffixes before using ndiff, so this doesn't seem that bad to me. Do others think this is acceptable? This whole issue made me wonder if it should be possible to disable the pytest_assert_binrepr hook. It is probably sufficient to add an option which can be checked right at the start of the builtin pytest_assert_binrepr hook which will just return None or [] right away. But what is the best way to access the config setting from there? Is it fine to use py.test.config or should the setting be piggy-backed on some global again (e.g. change the current py.builtin.builtins.AssertionError._pytesthook into a ._pytestconfig)? Or maybe there's a more elegant solution for this that I'm unaware of? Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From holger at merlinux.eu Wed Sep 29 15:28:18 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 29 Sep 2010 15:28:18 +0200 Subject: [py-dev] controlling assertions and difflib In-Reply-To: References: Message-ID: <20100929132818.GK20695@trillke.net> Hey Floris, all, On Wed, Sep 29, 2010 at 00:12 +0100, Floris Bruynooghe wrote: > As Holger pointed out the following test hangs py.test in my assertion > clone, it triggers a bug in difflib: > > def test_hang_in_diff(): > x = "1\n" * 1000 + "abc" + "2\n" * 1000 > y = "1\n" * 1000 + "def" + "2\n" * 1000 > assert x == y > > It seems difflib can not deal with the 1000 identical trailing items. > > It fails with a TypeError somewhere very deep in a recursion or, when > consuming the generator inside a list comprehension, just completely > hangs. Also before failing it considers all the 2s as different too. > Rather then debugging difflib (there are several possible bugs on > bugs.python.org which might relate to this) I've come up with some > code that does produce this instead: > > ______________________________ test_hang_in_diff _______________________________ > /tmp/sandbox/test_bugs.py:4: in test_hang_in_diff > > assert x == y > E assert '1\n1\n1\n1\n...n2\n2\n2\n2\n' == '1\n1\n1\n1\n...n2\n2\n2\n2\n' > E Skipping 1990 identical leading characters in diff > E Skipping 1991 identical trailing characters in diff > E 1 > E 1 > E 1 > E 1 > E 1 > E - abc2 > E + def2 > E 2 > E 2 > E 2 > E 2 looks good to me! > Another solution is to use context_diff() which does not have this > problem, but for some reason also considers all the trailing 2s to be > different. Additionally context diff also doesn't provide the nice > markers for differences inside a line which might be problematic as > there's no guarantee there will be nice line endings in the input. > > In general I was already thinking of cutting off similar prefixes and > suffixes before using ndiff, so this doesn't seem that bad to me. Do > others think this is acceptable? I find it acceptable. I guess we need to see how it works in RL projects. Btw, I find it hard to guess how many people actually read this ML, despite there being some >130 subscribers. But we'll get issue reports when things go bad i guess :) > This whole issue made me wonder if it should be possible to disable > the pytest_assert_binrepr hook. We could do a general "--assertmode=(choice)" with e.g.: 0: (no) don't do any assert reinterp (like --no-assert now) 1: (basic) do basic reinterpreation 2: (advanced) do basic reinterpretation + customizing hooks and default to 2. > It is probably sufficient to add an > option which can be checked right at the start of the builtin > pytest_assert_binrepr hook which will just return None or [] right > away. But what is the best way to access the config setting from > there? Is it fine to use py.test.config or should the setting be > piggy-backed on some global again (e.g. change the current > py.builtin.builtins.AssertionError._pytesthook into a ._pytestconfig)? > Or maybe there's a more elegant solution for this that I'm unaware > of? I think there is an elegant solution. Instead of using "hook" in py/code/ we use a generic "customize" function that is to be called. In pytest_assertion.py's pytest_configure we write that function such that it calls pytest_assert_binrepr(config, ...) hope that makes sense. If in doubt leave it to me and just use py.test.config. I also want to introduce an "ini" style file for py.test, btw. hopefully real soon now. best, holger From issues-noreply at bitbucket.org Wed Sep 29 15:52:19 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Wed, 29 Sep 2010 13:52:19 -0000 Subject: [py-dev] New issue 123 in py-trunk: new invocation: "python -m py.test" Message-ID: <28eb92790aa9b9656133c9641ee1f4a1@bitbucket.org> --- you can reply above this line --- New issue 123: new invocation: "python -m py.test" http://bitbucket.org/hpk42/py-trunk/issue/123/new-invocation-python-m-pytest holger krekel / hpk42 on Wed, 29 Sep 2010 15:52:19 +0200: Description: As with unittest2 and other tools it'd be nice to be able to call py.test without invoking an extra command line tool. For example, on Windows/Jython setuptools/distutils does not even create working command line tool scripts (known issue). Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Thu Sep 30 16:33:58 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Thu, 30 Sep 2010 14:33:58 -0000 Subject: [py-dev] New issue 124 in py-trunk: disabling capture makes the capfd funcarg break py.test Message-ID: --- you can reply above this line --- New issue 124: disabling capture makes the capfd funcarg break py.test http://bitbucket.org/hpk42/py-trunk/issue/124/disabling-capture-makes-the-capfd-funcarg-break Ronny Pfannschmidt / RonnyPfannschmidt on Thu, 30 Sep 2010 16:33:58 +0200: Description: the bottom line is getting a invalid fd in the terminalwriter {{{ #!text $ py.test -v testing/ -k test_stdouterrin_setnull inserting into sys.path: /home/ronny/Projects/py ==================================================================== test session starts ==================================================================== platform linux2 -- Python 2.6.5 -- pytest-1.4.0a1 -- /usr/bin/python gateway test setup scope: session execnet: /home/ronny/Projects/execnet/execnet/__init__.pyc -- 1.0.9.dev0 test path 1: testing/ testing/test_basics.py:145: test_stdouterrin_setnull PASSED ==================================================== 324 tests deselected by 'test_stdouterrin_setnull' ===================================================== ========================================================= 1 passed, 324 deselected in 0.27 seconds ========================================================== 16:28:15 | ~/Projects/execnet $ py.test -v testing/ -ks test_stdouterrin_setnull inserting into sys.path: /home/ronny/Projects/py ==================================================================== test session starts ==================================================================== platform linux2 -- Python 2.6.5 -- pytest-1.4.0a1 -- /usr/bin/python gateway test setup scope: session execnet: /home/ronny/Projects/execnet/execnet/__init__.pyc -- 1.0.9.dev0 ERROR: file not found: /home/ronny/Projects/execnet/test_stdouterrin_setnull 16:28:20 | ~/Projects/execnet $ py.test -vs -k test_stdouterrin_setnull inserting into sys.path: /home/ronny/Projects/py ==================================================================== test session starts ==================================================================== platform linux2 -- Python 2.6.5 -- pytest-1.4.0a1 -- /usr/bin/python gateway test setup scope: session execnet: /home/ronny/Projects/execnet/execnet/__init__.pyc -- 1.0.9.dev0 test path 1: /home/ronny/Projects/execnet testing/test_basics.py:145: test_stdouterrin_setnull Traceback (most recent call last): File "/home/ronny/Projects/py/bin/py.test", line 3, in py.cmdline.pytest() File "/home/ronny/Projects/py/py/_cmdline/pytest.py", line 5, in main raise SystemExit(py.test.cmdline.main(args)) File "/home/ronny/Projects/py/py/_test/session.py", line 21, in main exitstatus = config.hook.pytest_cmdline_main(config=config) File "/home/ronny/Projects/py/py/_test/pluginmanager.py", line 352, in __call__ return self.hookrelay._performcall(self.name, mc) File "/home/ronny/Projects/py/py/_test/pluginmanager.py", line 338, in _performcall return multicall.execute() File "/home/ronny/Projects/py/py/_test/pluginmanager.py", line 244, in execute res = method(**kwargs) File "/home/ronny/Projects/py/py/_plugin/pytest_default.py", line 8, in pytest_cmdline_main return Session(config).main() File "/home/ronny/Projects/py/py/_test/session.py", line 77, in main self.config.pluginmanager.notify_exception(excinfo) File "/home/ronny/Projects/py/py/_test/pluginmanager.py", line 166, in notify_exception return self.hook.pytest_internalerror(excrepr=excrepr) File "/home/ronny/Projects/py/py/_test/pluginmanager.py", line 352, in __call__ return self.hookrelay._performcall(self.name, mc) File "/home/ronny/Projects/py/py/_test/pluginmanager.py", line 338, in _performcall return multicall.execute() File "/home/ronny/Projects/py/py/_test/pluginmanager.py", line 244, in execute res = method(**kwargs) File "/home/ronny/Projects/py/py/_plugin/pytest_terminal.py", line 120, in pytest_internalerror self.write_line("INTERNALERROR> " + line) File "/home/ronny/Projects/py/py/_plugin/pytest_terminal.py", line 112, in write_line self._tw.line(line, **markup) File "/home/ronny/Projects/py/py/_io/terminalwriter.py", line 182, in line self.write(s, **kw) File "/home/ronny/Projects/py/py/_io/terminalwriter.py", line 168, in write self._file.flush() IOError: [Errno 9] Bad file descriptor Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.