From sean.dawson at gmail.com Thu Aug 11 15:37:13 2005 From: sean.dawson at gmail.com (Sean Dawson) Date: Thu, 11 Aug 2005 14:37:13 +0100 Subject: [py-dev] PYTHONPATH not being honored Message-ID: <95A532F3-5BE5-4252-AEC0-B43B3818AAA0@gmail.com> Hello, I'm getting funky errors trying to set PYTHONPATH to override libraries. Consider the following example: $ cd sandbox $ mkdir trac $ touch trac/__init__.py $ vim test_pypath.py inside this file I just write: --- import trac print trac.__file__ assert 1==2 --- Note: I have a trac library installed in my site-packages directory. Running py.test in the sandbox directory gives the result I would expect, that is, it correctly imports the trac folder located in my sandbox dir. However, I'm experiencing problems when I do the following: $ mkdir blah $ mv trac blah/trac $ PYTHONPATH=blah py.test In this case, however, the print statement tells me the trac library from my site-packages is getting loaded. Any ideas? I am running py.dist trunk Thanks, Sean From ianb at colorstudy.com Fri Aug 12 21:35:10 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 12 Aug 2005 14:35:10 -0500 Subject: [py-dev] Re: [Distutils] setuptools presentation In-Reply-To: <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com> References: <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com> Message-ID: <42FCF9EE.6070401@colorstudy.com> CC'ing to bring together opposing viewpoints... (this is referring to setuptools' "setup.py test" command, which expects to have a TestSuite to run) Phillip J. Eby wrote: > Also, the 'test' command isn't "limited" to unittest, it > just expects a unittest-compatible wrapper. For example, doctest has > functions that wrap doctests as unittest suites, so it's quite possible > to use that. As far as I'm concerned, unittest suites are the WSGI of > testing; if somebody creates a fancy new test framework, they should > darn well make it possible to put a unittest suite wrapper around it, so > that people can integrate their existing tests. :) Well, I don't know what exactly my opinion is. At a py.test user I don't have TestSuites for my tests. I've argued py.test should load TestCase-based tests by default, but this is kind of the opposite. I don't think it would be that hard to produce such suites; the test items that py.test collects could just be stuffed into a TestCase. The result would be acceptable as a sort of "this package thinks it runs okay" test. It's not the frontend I'd like to give to users. That said, if "python setup.py test ..." was completely equivalent to "py.test ..." then that would be great, because though the interface would be different from projects that use unittest, the entry point would be the same (assuming py.test is installed). I suppose a package could add an entry point that overrides the normal setuptools test command...? -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pobrien at orbtech.com Fri Aug 12 22:06:27 2005 From: pobrien at orbtech.com (Patrick K. O'Brien) Date: Fri, 12 Aug 2005 15:06:27 -0500 Subject: [py-dev] Re: [Distutils] setuptools presentation In-Reply-To: <42FCF9EE.6070401@colorstudy.com> References: <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com> <42FCF9EE.6070401@colorstudy.com> Message-ID: <42FD0143.9020904@orbtech.com> Ian Bicking wrote: > CC'ing to bring together opposing viewpoints... (this is referring to > setuptools' "setup.py test" command, which expects to have a TestSuite > to run) > > Phillip J. Eby wrote: > >> Also, the 'test' command isn't "limited" to unittest, it just expects >> a unittest-compatible wrapper. For example, doctest has functions >> that wrap doctests as unittest suites, so it's quite possible to use >> that. As far as I'm concerned, unittest suites are the WSGI of >> testing; if somebody creates a fancy new test framework, they should >> darn well make it possible to put a unittest suite wrapper around it, >> so that people can integrate their existing tests. :) > > Well, I don't know what exactly my opinion is. At a py.test user I > don't have TestSuites for my tests. I've argued py.test should load > TestCase-based tests by default, but this is kind of the opposite. I > don't think it would be that hard to produce such suites; the test items > that py.test collects could just be stuffed into a TestCase. > > The result would be acceptable as a sort of "this package thinks it runs > okay" test. It's not the frontend I'd like to give to users. > > That said, if "python setup.py test ..." was completely equivalent to > "py.test ..." then that would be great, because though the interface > would be different from projects that use unittest, the entry point > would be the same (assuming py.test is installed). I suppose a package > could add an entry point that overrides the normal setuptools test > command...? +1 on the latter (make "python setup.py test ..." equivalent to "py.test ...") over the former. I'd hate to see py.test have to conform to the awkward conventions introduced by unittest. Also, the MEMS folks have that Sancho thing, which I seem to recall is a testing framework. Might want to get their opinion on that matter, as they use that testing convention for Durus, Quixote, and other packages. -- Patrick K. O'Brien Orbtech http://www.orbtech.com Schevo http://www.schevo.org Pypersyst http://www.pypersyst.org From hpk at trillke.net Fri Aug 12 22:10:22 2005 From: hpk at trillke.net (holger krekel) Date: Fri, 12 Aug 2005 22:10:22 +0200 Subject: [py-dev] Re: [Distutils] setuptools presentation In-Reply-To: <42FCF9EE.6070401@colorstudy.com> References: <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com> <42FCF9EE.6070401@colorstudy.com> Message-ID: <20050812201022.GC11996@solar.trillke.net> On Fri, Aug 12, 2005 at 14:35 -0500, Ian Bicking wrote: > CC'ing to bring together opposing viewpoints... (this is referring to > setuptools' "setup.py test" command, which expects to have a TestSuite > to run) > > Phillip J. Eby wrote: > >Also, the 'test' command isn't "limited" to unittest, it > >just expects a unittest-compatible wrapper. For example, doctest has > >functions that wrap doctests as unittest suites, so it's quite possible > >to use that. As far as I'm concerned, unittest suites are the WSGI of > >testing; if somebody creates a fancy new test framework, they should > >darn well make it possible to put a unittest suite wrapper around it, so > >that people can integrate their existing tests. :) > > Well, I don't know what exactly my opinion is. At a py.test user I > don't have TestSuites for my tests. I've argued py.test should load > TestCase-based tests by default, but this is kind of the opposite. I > don't think it would be that hard to produce such suites; the test items > that py.test collects could just be stuffed into a TestCase. I agree without having looked deeply into it. Providing TestSuite instances that adapt running tests in a simple way with part of the py.test machinery should be quite feasible. Would probably mean that some of py.test's execution features would be switched off (like stdout/stderr capturing). However, as Ian hints at, py.test itself is already a project independent entry point for running tests in a given directory or for a given project. > The result would be acceptable as a sort of "this package thinks it runs > okay" test. It's not the frontend I'd like to give to users. unsurprisingly, i agree :-) > That said, if "python setup.py test ..." was completely equivalent to > "py.test ..." then that would be great, because though the interface > would be different from projects that use unittest, the entry point > would be the same (assuming py.test is installed). would make sense, i think. > I suppose a package could add an entry point that overrides > the normal setuptools test command...? Yes, i guess something needs to be configurable there as far as i understand the situation. I also presume that "setup.py test" would allow a custom test method to actually perform the execution of tests, not just provide a TestSuite. If so, it should at best become a simple matter of how a package can signal to setuptools that it wants its tests to be handled by py.test in which case the work would be defered to the (neccessarily) installed py library when "setup.py test" is invoked. It shouldn't be required from each application to provide this glue code as it should be generic. actually, how does "setup.py test" work today and which applications/projects are integrated with it if i may ask? cheers, holger From ianb at colorstudy.com Fri Aug 12 22:30:33 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 12 Aug 2005 15:30:33 -0500 Subject: [py-dev] Re: [Distutils] setuptools presentation In-Reply-To: <20050812201022.GC11996@solar.trillke.net> References: <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com> <42FCF9EE.6070401@colorstudy.com> <20050812201022.GC11996@solar.trillke.net> Message-ID: <42FD06E9.70405@colorstudy.com> holger krekel wrote: >>Well, I don't know what exactly my opinion is. At a py.test user I >>don't have TestSuites for my tests. I've argued py.test should load >>TestCase-based tests by default, but this is kind of the opposite. I >>don't think it would be that hard to produce such suites; the test items >>that py.test collects could just be stuffed into a TestCase. > > > I agree without having looked deeply into it. Providing TestSuite > instances that adapt running tests in a simple way with part > of the py.test machinery should be quite feasible. Would probably > mean that some of py.test's execution features would be switched > off (like stdout/stderr capturing). > > However, as Ian hints at, py.test itself is already a project independent > entry point for running tests in a given directory or for a given project. > > >>The result would be acceptable as a sort of "this package thinks it runs >>okay" test. It's not the frontend I'd like to give to users. > > > unsurprisingly, i agree :-) Actually I mistyped -- for developers (or potential developers) I think py.test's full featureset and frontend should be available. However, for mere library users there's a use case where they want to simply know that the tests pass, that the package is ok; failures are not expected. At least, when I was talking about this with some Perl people, they specifically said they like that CPAN automatically runs tests when you download a package. I think a uniform command-line interface is nice in this case; the user is just look for "ok" or "not ok", they aren't looking for a development tool. And it's possible that "setup.py test" should be that uniform interface. That doesn't exclude the possibility of a separate command like "setup.py pytest". >>I suppose a package could add an entry point that overrides >>the normal setuptools test command...? > > Yes, i guess something needs to be configurable there as far as i > understand the situation. I also presume that "setup.py test" > would allow a custom test method to actually perform the execution > of tests, not just provide a TestSuite. > > If so, it should at best become a simple matter of how a > package can signal to setuptools that it wants its tests to be > handled by py.test in which case the work would be defered to > the (neccessarily) installed py library when "setup.py test" > is invoked. It shouldn't be required from each application > to provide this glue code as it should be generic. > > actually, how does "setup.py test" work today and which > applications/projects are integrated with it if i may ask? AFAIK no applications are integrated. Well, maybe PEAK... this is all rather new. It's documented here: http://peak.telecommunity.com/DevCenter/setuptools#test-build-package-and-run-a-unittest-suite However, I believe each command to setup.py is simply an "entry_point". I can't find documentation for that at the moment. But it's something like this in the setup.py file: entry_points={ 'distutils.commands': [ 'test=test_module:test_command', ], } And then that test_command object (a class) has a specific interface (based on the distutils.Command class). So potentially a py.test-using project could do: entry_points={ 'distutils.commands': [ 'test=mypkg.commands:test']} # in mypkg.commands: from py.test.disutils import test I think that would override "setup.py test" for just that package. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Fri Aug 12 22:50:38 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 12 Aug 2005 16:50:38 -0400 Subject: [py-dev] Re: [Distutils] setuptools presentation In-Reply-To: <42FD06E9.70405@colorstudy.com> References: <20050812201022.GC11996@solar.trillke.net> <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com> <42FCF9EE.6070401@colorstudy.com> <20050812201022.GC11996@solar.trillke.net> Message-ID: <5.1.1.6.0.20050812164016.02937150@mail.telecommunity.com> At 03:30 PM 8/12/2005 -0500, Ian Bicking wrote: >However, I believe each command to setup.py is simply an "entry_point". I >can't find documentation for that at the moment. But it's something like >this in the setup.py file: > > entry_points={ > 'distutils.commands': [ > 'test=test_module:test_command', > ], > } Yes, that's exactly it. >And then that test_command object (a class) has a specific interface >(based on the distutils.Command class). So potentially a py.test-using >project could do: > > entry_points={ > 'distutils.commands': [ > 'test=mypkg.commands:test']} > > # in mypkg.commands: > from py.test.disutils import test > > >I think that would override "setup.py test" for just that package. Actually, it wouldn't, because the first 'test' command defined on sys.path will get used. This doesn't mean that we can't have define test entry points, though, that the setuptools 'test' command would then use to run other testing frameworks. For example, the builtin 'test' command could look up a 'test' command in the just-built package and run that. The problem is that the master test command will have to define what options it takes, because distutils doesn't delegate option parsing to the commands. :( But setuptools does offer another solution. In py.test's you could create a 'py.test' setup command, and projects that want to use it can do: setup( ... setup_requires=['py-test>=someversion'], project_args_for_py_test_here="something", ) They can also define a project alias in setup.cfg: [aliases] test = py.test This will make "setup.py test" run the "py.test" command instead. But to me, the simple and obvious thing to do is to make test suites. If py.test exports a zero-argument function that finds tests in the current directory and returns a suite, then a py.test-using package need only set 'test_suite="py.test.get_suite"' in order to use it, with no need to define an alias, nor would py.test need to define another distutils command. And, for basic "does this work?" testing, users get a uniform interface for the 'test' command. From pje at telecommunity.com Fri Aug 12 23:14:38 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 12 Aug 2005 17:14:38 -0400 Subject: [py-dev] Re: [Distutils] setuptools presentation In-Reply-To: <20050812201022.GC11996@solar.trillke.net> References: <42FCF9EE.6070401@colorstudy.com> <5.1.1.6.0.20050812131559.025dbc50@mail.telecommunity.com> <42FCF9EE.6070401@colorstudy.com> Message-ID: <5.1.1.6.0.20050812163037.029a96f0@mail.telecommunity.com> At 10:10 PM 8/12/2005 +0200, holger krekel wrote: >I agree without having looked deeply into it. Providing TestSuite >instances that adapt running tests in a simple way with part >of the py.test machinery should be quite feasible. Would probably >mean that some of py.test's execution features would be switched >off (like stdout/stderr capturing). You would just need to have each test object do its capturing between its calls to result.startTest and result.stopTest - I don't see that it would require switching anything off entirely. Just treat each test as the place for setup/teardown of any special processing. You might take a look at the doctest module, which provides similar wrappers. Doctests of course capture stdout, but this is done within each "test" from the point of view of the unittest text or GUI test runners. >However, as Ian hints at, py.test itself is already a project independent >entry point for running tests in a given directory or for a given project. Many projects distinguish functional or acceptance tests from their unit tests, so running *all* the tests isn't necessarily a good thing. The point of the current "setup.py test" command in setuptools is to provide a way to define what the default on-installation tests are. The test command also allows you to specify a different starting point as well, with the -s option, so that you can tell a user to run a specific test or suite. > > That said, if "python setup.py test ..." was completely equivalent to > > "py.test ..." then that would be great, because though the interface > > would be different from projects that use unittest, the entry point > > would be the same (assuming py.test is installed). > >would make sense, i think. > > > I suppose a package could add an entry point that overrides > > the normal setuptools test command...? > >Yes, i guess something needs to be configurable there as far as i >understand the situation. I also presume that "setup.py test" >would allow a custom test method to actually perform the execution >of tests, not just provide a TestSuite. It certainly could be done that way; my take on it is that both of the test frameworks in the stdlib support providing TestSuites, making it appear to be the One Obvious Way to do it. Providing TestSuites means that your tests can be run in conjunction with unittest and doctest suites in any unittest-compatible test runner. >If so, it should at best become a simple matter of how a >package can signal to setuptools that it wants its tests to be >handled by py.test in which case the work would be defered to >the (neccessarily) installed py library when "setup.py test" >is invoked. It shouldn't be required from each application >to provide this glue code as it should be generic. If py.test exposes a zero-argument function returning a test suite constructed from the default py.test approach, then a user need only specify: test_suite = "py.test.default_suite" in their setup() in order to use it. >actually, how does "setup.py test" work today and which It loads the suite named in setup() or via the '-s' command line option (using the default unittest test loader), and runs the unittest text-based test runner on it. (Note that this means that project management tools (like an IDE) could also run a GUI test runner against the same suite -- something that's not trivially achievable with tests that aren't TestSuite-compatible.) >applications/projects are integrated with it if i may ask? The PEAK family of projects (including setuptools itself, PyProtocols, PEAK, RuleDispatch, etc.) have been using it as long as setuptools has existed (~18 months). I'm willing to be flexible about the exact machinery invoked for tests, but I'd very much like to encourage "third-party" test frameworks to consider integrating with unittest (and thereby with all the other test frameworks that do), rather than encourage web-style proliferation of incompatible frameworks. :) doctest already can be called from unittest, so it seems like a good example for other test frameworks to follow. And I know that for me at least, no other framework is going to lure me away from unittest unless it allows me to run its tests as suites; I have way too many unittest-based tests I'm not going to go back and port. (I never even bothered *trying* doctest until it was possible to integrate it with my existing unittest-based suites.) From alain.poirier at net-ng.com Wed Aug 24 13:36:51 2005 From: alain.poirier at net-ng.com (Alain Poirier) Date: Wed, 24 Aug 2005 13:36:51 +0200 Subject: [py-dev] Greenlet copy Message-ID: <200508241336.51443.alain.poirier@net-ng.com> Hi, Do you think it could be possible to add a copy method to the greenlets ? Regards From uche.ogbuji at fourthought.com Wed Aug 24 19:14:49 2005 From: uche.ogbuji at fourthought.com (Uche Ogbuji) Date: Wed, 24 Aug 2005 11:14:49 -0600 Subject: [py-dev] error building greenlets Message-ID: <1124903690.18767.440.camel@borgia> I can't get greenlets to build on Fedora Core 4 Linux + Python 2.4.1 (or Python 2.3.4. I keep getting the following error. I'm using a pull form Subversion this week. I've tried all sorts of fussing about with the full py library (py/env.py and all that) and all sorts of random tinkering, to no avail. I don't find anything on the error via Google. Any ideas? Traceback (most recent call last): File "test_generator.py", line 1, in ? from py.magic import greenlet File "/home/uogbuji/src/publiccvs/py-dist/py/initpkg.py", line 181, in __getattr__ result = self.__package__._resolve(extpy) File "/home/uogbuji/src/publiccvs/py-dist/py/initpkg.py", line 69, in _resolve implmodule = self._loadimpl(fspath[:-3]) File "/home/uogbuji/src/publiccvs/py-dist/py/initpkg.py", line 99, in _loadimpl return __import__(modpath, None, None, ['__doc__']) File "/home/uogbuji/src/publiccvs/py-dist/py/magic/greenlet.py", line 5, in ? greenlet = path.getpymodule().greenlet File "/home/uogbuji/src/publiccvs/py-dist/py/path/local/local.py", line 408, in getpymodule mod = make_module_from_c(self) File "/home/uogbuji/src/publiccvs/py-dist/py/misc/buildcmodule.py", line 66, in make_module_from_c raise RuntimeError("cannot compile %s: %s\n%s" % (cfile, e, RuntimeError: cannot compile /home/uogbuji/src/publiccvs/py-dist/py/c-extension/greenlet/greenlet.c: error: command 'gcc' failed with exit status 1 /home/uogbuji/src/publiccvs/py-dist/py/c-extension/greenlet/greenlet.c: In function ?g_switchstack?: /home/uogbuji/src/publiccvs/py-dist/py/c-extension/greenlet/greenlet.c:274: warning: control may reach end of non-void function ?slp_switch? being inlined /home/uogbuji/src/publiccvs/py-dist/py/c-extension/greenlet/switch_x86_unix.h:34: error: PIC register ?ebx? clobbered in ?asm? [1] http://www.xml.com/pub/at/24 -- Uche Ogbuji Fourthought, Inc. http://uche.ogbuji.net http://fourthought.com http://copia.ogbuji.net http://4Suite.org Use CSS to display XML, part 2 - http://www-128.ibm.com/developerworks/edu/x-dw-x-xmlcss2-i.html XML Output with 4Suite & Amara - http://www.xml.com/pub/a/2005/04/20/py-xml.html Use XSLT to prepare XML for import into OpenOffice Calc - http://www.ibm.com/developerworks/xml/library/x-oocalc/ Schema standardization for top-down semantic transparency - http://www-128.ibm.com/developerworks/xml/library/x-think31.html From alain.poirier at net-ng.com Wed Aug 24 23:31:57 2005 From: alain.poirier at net-ng.com (Alain Poirier) Date: Wed, 24 Aug 2005 23:31:57 +0200 Subject: [py-dev] error building greenlets In-Reply-To: <1124903690.18767.440.camel@borgia> References: <1124903690.18767.440.camel@borgia> Message-ID: <200508242331.57666.alain.poirier@net-ng.com> A code compiled with the -fPIC option must preserve the ebx register. So, if it's noted as "clobbered", the new versions of gcc emit an error. A 'push ebx' / 'pop ebx' pair need to be added into the slp_switch function of switch_x86_unix.h Le Mercredi 24 Ao?t 2005 19:14, Uche Ogbuji a ?crit : > I can't get greenlets to build on Fedora Core 4 Linux + Python 2.4.1 (or > Python 2.3.4. I keep getting the following error. I'm using a pull > form Subversion this week. I've tried all sorts of fussing about with > the full py library (py/env.py and all that) and all sorts of random > tinkering, to no avail. I don't find anything on the error via Google. > Any ideas? > > > Traceback (most recent call last): > File "test_generator.py", line 1, in ? > from py.magic import greenlet > File "/home/uogbuji/src/publiccvs/py-dist/py/initpkg.py", line 181, in > __getattr__ > result = self.__package__._resolve(extpy) > File "/home/uogbuji/src/publiccvs/py-dist/py/initpkg.py", line 69, in > _resolve implmodule = self._loadimpl(fspath[:-3]) > File "/home/uogbuji/src/publiccvs/py-dist/py/initpkg.py", line 99, in > _loadimpl > return __import__(modpath, None, None, ['__doc__']) > File "/home/uogbuji/src/publiccvs/py-dist/py/magic/greenlet.py", line > 5, in ? > greenlet = path.getpymodule().greenlet > File "/home/uogbuji/src/publiccvs/py-dist/py/path/local/local.py", > line 408, in getpymodule > mod = make_module_from_c(self) > File "/home/uogbuji/src/publiccvs/py-dist/py/misc/buildcmodule.py", > line 66, in make_module_from_c > raise RuntimeError("cannot compile %s: %s\n%s" % (cfile, e, > RuntimeError: cannot > compile > /home/uogbuji/src/publiccvs/py-dist/py/c-extension/greenlet/greenlet.c: > error: command 'gcc' failed with exit status 1 > /home/uogbuji/src/publiccvs/py-dist/py/c-extension/greenlet/greenlet.c: > In function ?g_switchstack?: > /home/uogbuji/src/publiccvs/py-dist/py/c-extension/greenlet/greenlet.c:27 >4: warning: control may reach end of non-void function ?slp_switch? being > inlined > /home/uogbuji/src/publiccvs/py-dist/py/c-extension/greenlet/switch_x86_un >ix.h:34: error: PIC register ?ebx? clobbered in ?asm? > > > [1] http://www.xml.com/pub/at/24 From ianb at colorstudy.com Thu Aug 25 18:32:47 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 25 Aug 2005 11:32:47 -0500 Subject: [py-dev] Re: [SQLObject] Re: Bug trying to access foreignKey objects for inherited tables. Patch for test attached In-Reply-To: <20050825140601.GA7423@phd.pp.ru> References: <4304A80B.9070602@async.com.br> <20050825140601.GA7423@phd.pp.ru> Message-ID: <430DF2AF.8010203@colorstudy.com> Oleg Broytmann wrote: > Hello! I've returned. > > On Thu, Aug 18, 2005 at 12:23:55PM -0300, Evandro Vale Miquelito wrote: > >>Please apply this patch for the test suit and see the problem I got >>using revision 915. > > > I have a stupid problem with py.test - I was running it it with > "python -O" that disabled all assertions! I am so sorry for my stupidity. > Now, when I've fixed the problem I see that you don't need to patch > test_deep_inheritance.py - test_inherited_ForeignKey.py manifests the bug. > > I spend a few hours comparing revision 850 (that works) and the trunk > and finally found the difference (though didn't get the reason). I just > commited a fix that brings inherited ForeignKeys back. Please test if the > revision 944 fixes the problem and does not break anything else. Maybe py.test can test for this case (python -O), since certainly it's no good to run py.test with assertions disabled. Well, I guess you could possibly want to run the tests without assertions; should py.test emit a warning? Maybe just a little something when py.test is started up: try: assert 0 except AssertionError: pass else: print >> sys.stderr, 'Warning: Assertions are turned off! (python -O?)' Just while I'm looking at the code, would it be good in py.tests.cmdline.main(), when exiting with KeyboardInterrupt and verbose off, to do print "KeyboardInterrupt (-v to show traceback)" ? Suggestive error messages and all. I don't know if that assertion test would go in main() or elsewhere. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From hpk at trillke.net Thu Aug 25 18:54:16 2005 From: hpk at trillke.net (holger krekel) Date: Thu, 25 Aug 2005 18:54:16 +0200 Subject: [py-dev] Re: [SQLObject] Re: Bug trying to access foreignKey objects for inherited tables. Patch for test attached In-Reply-To: <430DF2AF.8010203@colorstudy.com> References: <4304A80B.9070602@async.com.br> <20050825140601.GA7423@phd.pp.ru> <430DF2AF.8010203@colorstudy.com> Message-ID: <20050825165416.GS666@solar.trillke.net> On Thu, Aug 25, 2005 at 11:32 -0500, Ian Bicking wrote: > Oleg Broytmann wrote: > >Hello! I've returned. > > > >On Thu, Aug 18, 2005 at 12:23:55PM -0300, Evandro Vale Miquelito wrote: > > > >>Please apply this patch for the test suit and see the problem I got > >>using revision 915. > > > > > > I have a stupid problem with py.test - I was running it it with > >"python -O" that disabled all assertions! I am so sorry for my stupidity. > > Now, when I've fixed the problem I see that you don't need to patch > >test_deep_inheritance.py - test_inherited_ForeignKey.py manifests the bug. > > > > I spend a few hours comparing revision 850 (that works) and the trunk > >and finally found the difference (though didn't get the reason). I just > >commited a fix that brings inherited ForeignKeys back. Please test if the > >revision 944 fixes the problem and does not break anything else. > > Maybe py.test can test for this case (python -O), since certainly it's > no good to run py.test with assertions disabled. Well, I guess you > could possibly want to run the tests without assertions; should py.test > emit a warning? Maybe just a little something when py.test is started up: > > try: > assert 0 > except AssertionError: > pass > else: > print >> sys.stderr, 'Warning: Assertions are turned off! (python -O?)' > > Just while I'm looking at the code, would it be good in > py.tests.cmdline.main(), when exiting with KeyboardInterrupt and verbose > off, to do print "KeyboardInterrupt (-v to show traceback)" ? > Suggestive error messages and all. I don't know if that assertion test > would go in main() or elsewhere. i added your two suggestions main(). Ian, for changes like this you can usually just go ahead and commit them. I can still correct them if neccessary. holger From arigo at tunes.org Sat Aug 27 18:27:26 2005 From: arigo at tunes.org (Armin Rigo) Date: Sat, 27 Aug 2005 18:27:26 +0200 Subject: [py-dev] Greenlet copy In-Reply-To: <200508241336.51443.alain.poirier@net-ng.com> References: <200508241336.51443.alain.poirier@net-ng.com> Message-ID: <20050827162726.GA4270@code1.codespeak.net> Hi Alain, On Wed, Aug 24, 2005 at 01:36:51PM +0200, Alain Poirier wrote: > Do you think it could be possible to add a copy method to > the greenlets ? To duplicate a greenlet, with each copy continuing to run from the same place? No; this would be full continuations, which isn't really possible with the current approach. Armin From arigo at tunes.org Sat Aug 27 18:31:00 2005 From: arigo at tunes.org (Armin Rigo) Date: Sat, 27 Aug 2005 18:31:00 +0200 Subject: [py-dev] greenlet.main_greenlet In-Reply-To: <428976CC.2040706@livelogix.com> References: <428976CC.2040706@livelogix.com> Message-ID: <20050827163100.GB4270@code1.codespeak.net> Hi Tom, On Tue, May 17, 2005 at 10:15:00AM +0530, Tom Locke wrote: > I had a need to be able to test if the current greenlet was the main > greenlet. I figured this might be something that crops up now and again, > so I added a module attribute `main_greeenlet` (also copied to the type > object), so I can just test for: > > greenlet.getcurrent() == greenlet.main_greenlet This won't work in a multithreaded application, because each thread has its own main greenlet. Instead, just check if greenlet.getcurrent() has a parent or not. All greenlets have a parent apart from the main one. Armin From arigo at tunes.org Sat Aug 27 18:39:54 2005 From: arigo at tunes.org (Armin Rigo) Date: Sat, 27 Aug 2005 18:39:54 +0200 Subject: [py-dev] error building greenlets In-Reply-To: <200508242331.57666.alain.poirier@net-ng.com> References: <1124903690.18767.440.camel@borgia> <200508242331.57666.alain.poirier@net-ng.com> Message-ID: <20050827163954.GC4270@code1.codespeak.net> Hi Alain, On Wed, Aug 24, 2005 at 11:31:57PM +0200, Alain Poirier wrote: > A code compiled with the -fPIC option must preserve the ebx register. Indeed. > So, if > it's noted as "clobbered", the new versions of gcc emit an error. This implication is not clear to me. The goal of the "clobbered" is precisely to force gcc to save the register to the stack and restore it. The fact that newer versions of gcc doesn't like 'ebx' in the clobbered list any more is an annoyance. > A 'push ebx' / 'pop ebx' pair need to be added into the slp_switch function > of switch_x86_unix.h Can you show the precise patch? I guess that you have in mind a "push ebx" in the first __asm__ line, and a "pop ebx" in the last one. This might be doing the trick but it looks very fragile (i.e. depending on the exact way gcc generates code). Now, I don't expect the value of the PIC register "ebx" to actually change during the program execution, so that it will have the same value in all greenlets at any time. If so, we don't actually have to save and restore it. Just removing the "ebx" in the two clobbered lists should be enough. This is the conclusion to which I came some time ago already, but the problem with it is to detect if the code is indeed being compiled with -fPIC or not. If not, the "ebx" in the clobbered is essential. Any clue? A bientot, Armin From alain.poirier at net-ng.com Sun Aug 28 15:08:56 2005 From: alain.poirier at net-ng.com (Alain Poirier) Date: Sun, 28 Aug 2005 15:08:56 +0200 Subject: [py-dev] error building greenlets In-Reply-To: <20050827163954.GC4270@code1.codespeak.net> References: <1124903690.18767.440.camel@borgia> <200508242331.57666.alain.poirier@net-ng.com> <20050827163954.GC4270@code1.codespeak.net> Message-ID: <200508281508.56861.alain.poirier@net-ng.com> Hi Armin, > On Wed, Aug 24, 2005 at 11:31:57PM +0200, Alain Poirier wrote: > > A code compiled with the -fPIC option must preserve the ebx register. > > Indeed. > > > So, if it's noted as "clobbered", the new versions of gcc emit an error. > > This implication is not clear to me. The goal of the "clobbered" is > precisely to force gcc to save the register to the stack and restore it. > The fact that newer versions of gcc doesn't like 'ebx' in the clobbered > list any more is an annoyance. From the gcc doc (and the tests I did), I understand that the clobbered list only indicate to the compiler what registers are used into the function, but not to save them. http://www.ibiblio.org/gferg/ldp/GCC-Inline-Assembly-HOWTO.html#ss5.3 > > A 'push ebx' / 'pop ebx' pair need to be added into the slp_switch > > function of switch_x86_unix.h > > Can you show the precise patch? I guess that you have in mind a "push > ebx" in the first __asm__ line, and a "pop ebx" in the last one. Yes > This might be doing the trick but it looks very fragile (i.e. depending on > the exact way gcc generates code). In theorie, we need to save "ebx" which is a bit annoying because the C macro SLP_SAVE_STATE do some 'return' and, in these cases, as ebx is not popped => crash. So we could change the 'return' by a 'goto end, in greenlet.c : #define SLP_SAVE_STATE(stackref, stsizediff) \ stackref += STACK_MAGIC; \ if (slp_save_state((char*)stackref)) { r = -1; goto end; } \ if (!PyGreen_ACTIVE(ts_target)) { r = 1; goto end; } \ stsizediff = ts_target->stack_start - (char*)stackref In switch_x86_unix.h (and 'end: ...' needed for all the other architectures :( : static int slp_switch(void) { register int *stackref, stsizediff; int r; __asm__ volatile ("" : : : "esi", "edi"); __asm__ ("push %ebx"); __asm__ ("movl %%esp, %0" : "=g" (stackref)); { SLP_SAVE_STATE(stackref, stsizediff); __asm__ volatile ( "addl %0, %%esp\n" "addl %0, %%ebp\n" : : "r" (stsizediff) ); SLP_RESTORE_STATE(); r = 0; } end: __asm__ ("pop %ebx"); return r; __asm__ volatile ("" : : : "esi", "edi"); } But, > Now, I don't expect the value of the PIC register "ebx" to actually > change during the program execution, so that it will have the same value > in all greenlets at any time. If so, we don't actually have to save and > restore it. Just removing the "ebx" in the two clobbered lists should > be enough. Right. As the "ebx" register is not modified by the asm of slp_switch, it stays constant. So, in practice, remove "ebx" from the clobbered list is enough and the simplest solution :) > This is the conclusion to which I came some time ago already, but the > problem with it is to detect if the code is indeed being compiled with > -fPIC or not. If not, the "ebx" in the clobbered is essential. Any > clue? Hmm, I don't see why "ebx" in the clobbered is essential for a code compiled without -fPIC ? Amicalement, Alain From bob at redivi.com Sun Aug 28 23:31:36 2005 From: bob at redivi.com (Bob Ippolito) Date: Sun, 28 Aug 2005 14:31:36 -0700 Subject: [py-dev] error building greenlets In-Reply-To: <20050827163954.GC4270@code1.codespeak.net> References: <1124903690.18767.440.camel@borgia> <200508242331.57666.alain.poirier@net-ng.com> <20050827163954.GC4270@code1.codespeak.net> Message-ID: <0F6893A4-EB56-4259-B448-C0E7EB85B1C3@redivi.com> On Aug 27, 2005, at 9:39 AM, Armin Rigo wrote: > Hi Alain, > > On Wed, Aug 24, 2005 at 11:31:57PM +0200, Alain Poirier wrote: > >> A code compiled with the -fPIC option must preserve the ebx register. >> > > Indeed. > > >> So, if >> it's noted as "clobbered", the new versions of gcc emit an error. >> > > This implication is not clear to me. The goal of the "clobbered" is > precisely to force gcc to save the register to the stack and > restore it. > The fact that newer versions of gcc doesn't like 'ebx' in the > clobbered > list any more is an annoyance. gcc4 also complains about this with r31 on PPC, in switch_ppc_macosx.h anyway. Removing it from the clobbered list doesn't seem to make any difference in the generated code. I'm not sure the PIC register needs to be preserved anyway, if it got mangled then it should crash when trying to call slp_restore_state. -bob From arigo at tunes.org Mon Aug 29 11:53:01 2005 From: arigo at tunes.org (Armin Rigo) Date: Mon, 29 Aug 2005 11:53:01 +0200 Subject: [py-dev] error building greenlets In-Reply-To: <200508281508.56861.alain.poirier@net-ng.com> References: <1124903690.18767.440.camel@borgia> <200508242331.57666.alain.poirier@net-ng.com> <20050827163954.GC4270@code1.codespeak.net> <200508281508.56861.alain.poirier@net-ng.com> Message-ID: <20050829095301.GA24462@code1.codespeak.net> Hi Alain & Bob, On Sun, Aug 28, 2005 at 03:08:56PM +0200, Alain Poirier wrote: > Hmm, I don't see why "ebx" in the clobbered is essential for a code compiled > without -fPIC ? I believe that 'ebx' is supposed to be a callee-saved register (like 'esi' and 'edi') in the non-PIC compilation modes. I have no clue about the PPC's 'r31' in non-PIC mode. Now maybe we can just forget about non-PIC modes for the greenlet extension module, with big warnings that this code needs to get the 'ebx' again if it is used in projects that are not compiled in this mode, like Stackless (where it comes from). A bientot, Armin From bob at redivi.com Tue Aug 30 22:55:29 2005 From: bob at redivi.com (Bob Ippolito) Date: Tue, 30 Aug 2005 13:55:29 -0700 Subject: [py-dev] error building greenlets In-Reply-To: <20050829095301.GA24462@code1.codespeak.net> References: <1124903690.18767.440.camel@borgia> <200508242331.57666.alain.poirier@net-ng.com> <20050827163954.GC4270@code1.codespeak.net> <200508281508.56861.alain.poirier@net-ng.com> <20050829095301.GA24462@code1.codespeak.net> Message-ID: On Aug 29, 2005, at 2:53 AM, Armin Rigo wrote: > Hi Alain & Bob, > > On Sun, Aug 28, 2005 at 03:08:56PM +0200, Alain Poirier wrote: > >> Hmm, I don't see why "ebx" in the clobbered is essential for a >> code compiled >> without -fPIC ? >> > > I believe that 'ebx' is supposed to be a callee-saved register > (like 'esi' > and 'edi') in the non-PIC compilation modes. > > I have no clue about the PPC's 'r31' in non-PIC mode. > > Now maybe we can just forget about non-PIC modes for the greenlet > extension module, with big warnings that this code needs to get the > 'ebx' again if it is used in projects that are not compiled in this > mode, like Stackless (where it comes from). On OS X, non-PIC is rarely ever used and wasn't supported at all until relatively recently (Mac OS X 10.3 / gcc 3.3, IIRC). I wouldn't worry about it. Even if it should be saved, a sane compiler simply isn't going to use a reserved register for that little chunk of code where there's so many free GPRs to choose from. It's especially unimportant for the greenlet extension module, of course, because you'd have to compile it statically into Python with - mdynamic-no-pic in order for it to matter. -bob From arigo at tunes.org Wed Aug 31 14:45:07 2005 From: arigo at tunes.org (Armin Rigo) Date: Wed, 31 Aug 2005 14:45:07 +0200 Subject: [py-dev] error building greenlets In-Reply-To: References: <1124903690.18767.440.camel@borgia> <200508242331.57666.alain.poirier@net-ng.com> <20050827163954.GC4270@code1.codespeak.net> <200508281508.56861.alain.poirier@net-ng.com> <20050829095301.GA24462@code1.codespeak.net> Message-ID: <20050831124506.GA20768@code1.codespeak.net> Hi Bob, On Tue, Aug 30, 2005 at 01:55:29PM -0700, Bob Ippolito wrote: > It's especially unimportant for the greenlet extension module, of > course, because you'd have to compile it statically into Python with - > mdynamic-no-pic in order for it to matter. Ok, so I guess I will remove "ebx" from the Unix PC include file and "r31" from both the MacOSX and Unix PPC target -- is that right? Armin From alain.poirier at net-ng.com Wed Aug 31 14:47:29 2005 From: alain.poirier at net-ng.com (Alain Poirier) Date: Wed, 31 Aug 2005 14:47:29 +0200 Subject: [py-dev] error building greenlets In-Reply-To: <20050831124506.GA20768@code1.codespeak.net> References: <1124903690.18767.440.camel@borgia> <20050831124506.GA20768@code1.codespeak.net> Message-ID: <200508311447.29353.alain.poirier@net-ng.com> Le Mercredi 31 Ao?t 2005 14:45, Armin Rigo a ?crit : > Hi Bob, > > On Tue, Aug 30, 2005 at 01:55:29PM -0700, Bob Ippolito wrote: > > It's especially unimportant for the greenlet extension module, of > > course, because you'd have to compile it statically into Python with - > > mdynamic-no-pic in order for it to matter. > > Ok, so I guess I will remove "ebx" from the Unix PC include file and > "r31" from both the MacOSX and Unix PPC target -- is that right? To me, ok for the "ebx" removal