From J.Generowicz at soton.ac.uk Wed Oct 10 02:05:00 2007 From: J.Generowicz at soton.ac.uk (Jacek Generowicz) Date: Wed, 10 Oct 2007 01:05:00 +0100 Subject: [py-dev] py.test --exec Message-ID: <19A88AC8-11BE-4C59-BDDF-33521749919B@soton.ac.uk> I am working with some code which, for MPI-related reasons, is constructed around a custom-built Python interpreter. I am trying to write some tests for this code using py.test. Trying to use the --exec option to instruct py.test to use the custom- built Python interpreter in question, results in the following failure. $ py.test interface/nmag --exec=~/nsim/pyfem3/pyfem3 inserting into sys.path: /home/jacek/nsim/py Traceback (most recent call last): File "/home/jacek/nsim/py/py/bin/py.test", line 4, in ? py.test.cmdline.main() File "/home/jacek/nsim/py/py/test/cmdline.py", line 13, in main session = config.initsession() File "/home/jacek/nsim/py/py/test/config.py", line 140, in initsession session = cls(self) File "/home/jacek/nsim/py/py/test/terminal/remote.py", line 58, in __init__ self._setexecutable() File "/home/jacek/nsim/py/py/test/terminal/remote.py", line 70, in _setexecutable assert executable is not None, executable AssertionError Looking at the source code, it is far from clear to me what the problem might be. What sort of values of --exec should I expect to work/fail? Thanks. P.S. I'm puzzled by the purpose of ", executable" in line 70 of remote.py: when the assertion fails, executable will always be None, so no message will ever appear. From fijal at genesilico.pl Wed Oct 10 08:58:27 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Wed, 10 Oct 2007 08:58:27 +0200 Subject: [py-dev] py.test --exec In-Reply-To: <19A88AC8-11BE-4C59-BDDF-33521749919B@soton.ac.uk> References: <19A88AC8-11BE-4C59-BDDF-33521749919B@soton.ac.uk> Message-ID: <470C7813.2010109@genesilico.pl> Jacek Generowicz wrote: > I am working with some code which, for MPI-related reasons, is > constructed around a custom-built Python interpreter. I am trying to > write some tests for this code using py.test. > > Trying to use the --exec option to instruct py.test to use the custom- > built Python interpreter in question, results in the following failure. > > $ py.test interface/nmag --exec=~/nsim/pyfem3/pyfem3 > inserting into sys.path: /home/jacek/nsim/py > Traceback (most recent call last): > File "/home/jacek/nsim/py/py/bin/py.test", line 4, in ? > py.test.cmdline.main() > File "/home/jacek/nsim/py/py/test/cmdline.py", line 13, in main > session = config.initsession() > File "/home/jacek/nsim/py/py/test/config.py", line 140, in > initsession > session = cls(self) > File "/home/jacek/nsim/py/py/test/terminal/remote.py", line 58, in > __init__ > self._setexecutable() > File "/home/jacek/nsim/py/py/test/terminal/remote.py", line 70, in > _setexecutable > assert executable is not None, executable > AssertionError > > Looking at the source code, it is far from clear to me what the > problem might be. What sort of values of --exec should I expect to > work/fail? > should expect python executable. I happily use --exec like --exec=python2.4, --exec=pypy-c. Another way to run it is to run py.test under custom executable (we use this with pypy-c), like ~/nsim/pyfem3/pyfem3 py.test interface/nmag Also my wild guess is that ~ does not get interpreted correctly, because shell leaves it alone. Try /home/jacek instead. > Thanks. > > P.S. I'm puzzled by the purpose of ", executable" in line 70 of > remote.py: when the assertion fails, executable will always be None, > so no message will ever appear. > I'm puzzled as well :) Cheers, fijal :. From J.Generowicz at soton.ac.uk Wed Oct 10 12:43:26 2007 From: J.Generowicz at soton.ac.uk (Jacek Generowicz) Date: Wed, 10 Oct 2007 11:43:26 +0100 Subject: [py-dev] py.test --exec In-Reply-To: <470C7813.2010109@genesilico.pl> References: <19A88AC8-11BE-4C59-BDDF-33521749919B@soton.ac.uk> <470C7813.2010109@genesilico.pl> Message-ID: <70CCFF9C-3DE6-4EBE-8845-7C3C771E215B@soton.ac.uk> On 10 Oct 2007, at 07:58, Maciek Fijalkowski wrote: > Jacek Generowicz wrote: >> What sort of values of --exec should I expect to work/fail? >> > should expect python executable. I happily use --exec like > --exec=python2.4, --exec=pypy-c. Sure. It works just fine for me with standard Python interpreters, just not the custom-build. > Another way to run it is to run py.test > under custom executable (we use this with pypy-c), like > > ~/nsim/pyfem3/pyfem3 py.test interface/nmag $ pyfem3/pyfem3 py/py/bin/py.test interface/nmag Traceback (most recent call last): File "", line 1, in ? File "py/py/bin/py.test", line 3, in ? from _findpy import py ImportError: No module named _findpy $ echo $PYTHONPATH /home/jacek/nsim/py This PYTHONPATH was set by eval `python nsim/py/py/env.py`. However, if I manually add /home/jacek/nsim/py/py/bin to the PYTHONPATH, I get more success. Should env.py be adjusted to include py/py/bin in the PYTHONPATH ? Now I am left with interesting interactions between the namespace magic in py.test and the namespace and command line parsing magic in the custom-built Python, but I'll have to do quite a bit of poking around to understand exactly what is going on. Anyway, your suggestion has got me quite a but further. Thanks. From fijal at genesilico.pl Wed Oct 10 13:56:18 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Wed, 10 Oct 2007 13:56:18 +0200 Subject: [py-dev] py.test --exec In-Reply-To: <70CCFF9C-3DE6-4EBE-8845-7C3C771E215B@soton.ac.uk> References: <19A88AC8-11BE-4C59-BDDF-33521749919B@soton.ac.uk> <470C7813.2010109@genesilico.pl> <70CCFF9C-3DE6-4EBE-8845-7C3C771E215B@soton.ac.uk> Message-ID: <470CBDE2.1010703@genesilico.pl> Jacek Generowicz wrote: > > On 10 Oct 2007, at 07:58, Maciek Fijalkowski wrote: > >> Jacek Generowicz wrote: >>> What sort of values of --exec should I expect to work/fail? >>> >> should expect python executable. I happily use --exec like >> --exec=python2.4, --exec=pypy-c. > > Sure. It works just fine for me with standard Python interpreters, > just not the custom-build. > >> Another way to run it is to run py.test >> under custom executable (we use this with pypy-c), like >> >> ~/nsim/pyfem3/pyfem3 py.test interface/nmag > > $ pyfem3/pyfem3 py/py/bin/py.test interface/nmag > Traceback (most recent call last): > File "", line 1, in ? > File "py/py/bin/py.test", line 3, in ? > from _findpy import py > ImportError: No module named _findpy > > $ echo $PYTHONPATH > /home/jacek/nsim/py > > This PYTHONPATH was set by eval `python nsim/py/py/env.py`. > > However, if I manually add /home/jacek/nsim/py/py/bin to the > PYTHONPATH, I get more success. > > Should env.py be adjusted to include py/py/bin in the PYTHONPATH ? Well, that's another issue, also related to your custom interpreter. You should have py lib on path in order to run py.test. Whether it's done by symlinking it somewhere or by adding to PYTHONPATH, doesn't matter. You might also encounter the same when trying to run eg django on top of your python. You might trick site.py to include default python path (/usr/lib/python2.x/site-packages for example) in your custom interpreter to do that. > > Now I am left with interesting interactions between the namespace > magic in py.test and the namespace and command line parsing magic in > the custom-built Python, but I'll have to do quite a bit of poking > around to understand exactly what is going on. pylib should be very careful about populating namespace. everything is under py.xxx, or py.__.xxx, so shouldn't be a problem. More mess is with builtin modules like code.py etc., make sure you have no such files in current directory, or available around. Cheers, fijal > > Anyway, your suggestion has got me quite a but further. > > Thanks. You're welcome. For quicker answers, go to #pylib on irc.freenode.net. Cheers, fijal :. From J.Generowicz at soton.ac.uk Fri Oct 12 23:08:55 2007 From: J.Generowicz at soton.ac.uk (Jacek Generowicz) Date: Fri, 12 Oct 2007 22:08:55 +0100 Subject: [py-dev] --pdb option changes success of test Message-ID: <10866A3D-6A37-4302-A92D-B0BA3A18AF3D@soton.ac.uk> The following test def test_import_ipython_shell(): import IPython assert 'IPython' in dir() fails if run without --pdb, but succeeds if run with --pdb. (I'm using py-lib checked out from http://codespeak.net/svn/py/dist about 10 minutes ago and IPython 0.7.2) Any ideas? From fijal at genesilico.pl Tue Oct 16 09:36:44 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Tue, 16 Oct 2007 09:36:44 +0200 Subject: [py-dev] --pdb option changes success of test In-Reply-To: <10866A3D-6A37-4302-A92D-B0BA3A18AF3D@soton.ac.uk> References: <10866A3D-6A37-4302-A92D-B0BA3A18AF3D@soton.ac.uk> Message-ID: <47146A0C.7080104@genesilico.pl> Jacek Generowicz wrote: > The following test > > def test_import_ipython_shell(): > import IPython > assert 'IPython' in dir() > > fails if run without --pdb, but succeeds if run with --pdb. > > (I'm using py-lib checked out from http://codespeak.net/svn/py/dist > about 10 minutes ago and IPython 0.7.2) > > Any ideas? Honestly, no. Isn't Ipython doing some magic with __import__? It might interact with pylib magic __import__. Also input catching might interact. Try with -s and see if this one goes. Cheers, fijal :. From ondrej at certik.cz Thu Oct 18 23:38:19 2007 From: ondrej at certik.cz (Ondrej Certik) Date: Thu, 18 Oct 2007 23:38:19 +0200 Subject: [py-dev] [pypy-dev] py.test bisection algorithm In-Reply-To: <4717C50B.2080404@genesilico.pl> References: <85b5c3130710111729h62c1257ga268e432851ddea@mail.gmail.com> <4717C50B.2080404@genesilico.pl> Message-ID: <85b5c3130710181438q1ad40f22q8f48de328f886dc6@mail.gmail.com> Awesome, thanks a lot. Let's do it soon. There are more things - Kirill (another developer of SymPy) implemented a new feature: http://code.google.com/p/sympy/issues/detail?id=389 the relevant file is here: http://sympy.googlecode.com/svn/trunk/sympy/utilities/pytest.py and we would like to get it integrated to py.test. It works really well for us in the serial mode, but it doesn't yet work in the "py.test -d" mode. And now I found it doesn't work in the boxed mode, so it still needs some work. Another problem: "py.test -w" doesn't work in Debian, I reported a bug long time ago: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=434226 But got no response. I am CCing to the Debian maintainer - if you don't have time to maintain the package, I can take over it, I have several python related packages in Debian already and I need the py.test to work correctly if possible. Another problem with "py.test -d" is that it fails for sympy, but "py.test" only works. This is related to my last email, so by implementing bisect at least in serial mode should help me to trace all those nasty bugs in sympy down. One nice feature would be for py.test to remember the order of tests in "py.test -d" - and if it fails, but succeeds in the serial mode, it should automatically bisect and tell me - hey, this test works fine, but if executed just after that test, it fails. Ondrej On 10/18/07, Maciek Fijalkowski wrote: > Ondrej Certik wrote: > > Hi, > > > > we are using py.test in SymPy and unfortunately, we have some bugs in > > SymPy, that occurs when certain tests are run and don't occur when the > > tests are run in different order. It's very time consuming to > > determine which tests cause the problem (I did that several times by > > hand). It occured to me that it should be possible to enhance py.test > > by this facility to do it automatically. > > > > Example: > > > > 1) This passes: > > > > $ py.test sympy/series/tests/test_series.py -k issue409 > > > > This doesn't > > > > 2) $ py.test > > > > One problem is, that 2) runs for several minutes, another problem is > > that I need to specify all tests on the command line and then deleting > > some of them it see if it still fails, until I narrow the issue down, > > usually quite nicely. Then I need to play with the "-k" parameter to > > try different test cases in the file, until I determine the minimal > > set of tests that, when executed in order, produce the error. > > > > This can by done automatically - the py.test will be given a set of > > files (or just tests) that pass and another set that fails and it will > > narrow the problem down, by bisecting. > > > > I would like to implement this in py.test. So I'll try my best and > > send you a patch. If you have some ideas, that could help me, I am > > interested. > > > > Thanks a lot, > > Ondrej > Hi Ondrej. > > Sorry for late reply. This mail should go to py-dev instead or at least > also. This would cause faster reply (still my fault, I was offline for a > while :) > > Anyway - very nice idea. The long-term goal for py.test is to support > this algorithm ie. in respect to revisions (give me revision that broke > test X) etc. So if you go for implementing this, would be very cool for > many things. Lately I didn't invest too much time in py.test > developement, but I would be really happy to review your patches and/or > help you. You can find help most of the time on #pylib somehow (#pypy > might work as well in case noone hangs around #pylib). > > Cheers, > fijal > > > :. > > On 10/18/07, Maciek Fijalkowski wrote: > Ondrej Certik wrote: > > > > > > > > Oh, and by the way. We've got boxed version of py.test (--box or sth, > read --help), which forks for every test, so you're sure that every test > is run in separate environment. (That doesn't solve your problem, but > helps the other way around, when tests are run ok only when they're > together). > > Cheers, > fijal > > > :. > > From fijal at genesilico.pl Fri Oct 19 09:37:58 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Fri, 19 Oct 2007 09:37:58 +0200 Subject: [py-dev] [pypy-dev] py.test bisection algorithm In-Reply-To: <85b5c3130710181438q1ad40f22q8f48de328f886dc6@mail.gmail.com> References: <85b5c3130710111729h62c1257ga268e432851ddea@mail.gmail.com> <4717C50B.2080404@genesilico.pl> <85b5c3130710181438q1ad40f22q8f48de328f886dc6@mail.gmail.com> Message-ID: <47185ED6.70003@genesilico.pl> Ondrej Certik wrote: > Awesome, thanks a lot. Let's do it soon. There are more things - > Kirill (another developer of SymPy) implemented a new feature: > > http://code.google.com/p/sympy/issues/detail?id=389 > > the relevant file is here: > > http://sympy.googlecode.com/svn/trunk/sympy/utilities/pytest.py > > and we would like to get it integrated to py.test. It works really > well for us in the serial mode, but it doesn't yet work in the > "py.test -d" mode. And now I found it doesn't work in the boxed mode, > so it still needs some work. > > Another problem: "py.test -w" doesn't work in Debian, I reported a bug > long time ago: > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=434226 > > But got no response. I am CCing to the Debian maintainer - if you > don't have time to maintain the package, I can take over it, I have > several python related packages in Debian already and I need the > py.test to work correctly if possible. > > Another problem with "py.test -d" is that it fails for sympy, but > "py.test" only works. This is related to my last email, so by > implementing bisect at least in serial mode should help me to trace > all those nasty bugs in sympy down. One nice feature would be for > py.test to remember the order of tests in "py.test -d" - and if it > fails, but succeeds in the serial mode, it should automatically bisect > and tell me - hey, this test works fine, but if executed just after > that test, it fails. > > Ondrej Ok, I'll take a look. Would be cool to have nice and working debian package, but indeed we're not good in packaging. Also, we would like to do 1.0 release at some point. The main blocker is some refactorings to-be-done (some internal unification) and eventually unittest support and/or cross-platform testing (like -d, but run every test per every platform). Cheers, fijal :. From ondrej at certik.cz Fri Oct 19 09:44:56 2007 From: ondrej at certik.cz (Ondrej Certik) Date: Fri, 19 Oct 2007 09:44:56 +0200 Subject: [py-dev] [pypy-dev] py.test bisection algorithm In-Reply-To: <47185ED6.70003@genesilico.pl> References: <85b5c3130710111729h62c1257ga268e432851ddea@mail.gmail.com> <4717C50B.2080404@genesilico.pl> <85b5c3130710181438q1ad40f22q8f48de328f886dc6@mail.gmail.com> <47185ED6.70003@genesilico.pl> Message-ID: <85b5c3130710190044v182db0fem41b08d0ed6ee8288@mail.gmail.com> > Ok, I'll take a look. Would be cool to have nice and working debian > package, but indeed we're not good in packaging. Also, we would like to > do 1.0 release at some point. The main blocker is some refactorings > to-be-done (some internal unification) and eventually unittest support > and/or cross-platform testing (like -d, but run every test per every > platform). Guilherme wrote me that he orphaned the package, which means I can adopt it. So I'll try the svn and see if what works and what doesn't and then either I can package the svn version in Debian, or (preferably) you make a new release and I'll package that. Ondrej From fijal at genesilico.pl Fri Oct 19 09:52:11 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Fri, 19 Oct 2007 09:52:11 +0200 Subject: [py-dev] [pypy-dev] py.test bisection algorithm In-Reply-To: <85b5c3130710190044v182db0fem41b08d0ed6ee8288@mail.gmail.com> References: <85b5c3130710111729h62c1257ga268e432851ddea@mail.gmail.com> <4717C50B.2080404@genesilico.pl> <85b5c3130710181438q1ad40f22q8f48de328f886dc6@mail.gmail.com> <47185ED6.70003@genesilico.pl> <85b5c3130710190044v182db0fem41b08d0ed6ee8288@mail.gmail.com> Message-ID: <4718622B.60401@genesilico.pl> Ondrej Certik wrote: >> Ok, I'll take a look. Would be cool to have nice and working debian >> package, but indeed we're not good in packaging. Also, we would like to >> do 1.0 release at some point. The main blocker is some refactorings >> to-be-done (some internal unification) and eventually unittest support >> and/or cross-platform testing (like -d, but run every test per every >> platform). >> > > Guilherme wrote me that he orphaned the package, which means I can > adopt it. So I'll try the svn and see if what works and what doesn't > and then either I can package the svn version in Debian, or > (preferably) you make a new release and I'll package that. > > Ondrej > > :. > Not sure what is in the package, but the last release is 0.9 (I think package is like 0.7). Which means most of the stuff should work. I would not like to have svn package, we can always do 0.9.1 if you find nice stuff which works in svn, but doesn't in 0.9. But having 0.9 is still an improvement :-) Also we need to wait for holger to come back (should be soon). Cheers, fijal :. From ondrej at certik.cz Fri Oct 19 09:55:51 2007 From: ondrej at certik.cz (Ondrej Certik) Date: Fri, 19 Oct 2007 09:55:51 +0200 Subject: [py-dev] [pypy-dev] py.test bisection algorithm In-Reply-To: <4718622B.60401@genesilico.pl> References: <85b5c3130710111729h62c1257ga268e432851ddea@mail.gmail.com> <4717C50B.2080404@genesilico.pl> <85b5c3130710181438q1ad40f22q8f48de328f886dc6@mail.gmail.com> <47185ED6.70003@genesilico.pl> <85b5c3130710190044v182db0fem41b08d0ed6ee8288@mail.gmail.com> <4718622B.60401@genesilico.pl> Message-ID: <85b5c3130710190055t5d2f01f3q80b00b99316a5f61@mail.gmail.com> On 10/19/07, Maciek Fijalkowski wrote: > Ondrej Certik wrote: > >> Ok, I'll take a look. Would be cool to have nice and working debian > >> package, but indeed we're not good in packaging. Also, we would like to > >> do 1.0 release at some point. The main blocker is some refactorings > >> to-be-done (some internal unification) and eventually unittest support > >> and/or cross-platform testing (like -d, but run every test per every > >> platform). > >> > > > > Guilherme wrote me that he orphaned the package, which means I can > > adopt it. So I'll try the svn and see if what works and what doesn't > > and then either I can package the svn version in Debian, or > > (preferably) you make a new release and I'll package that. > > > > Ondrej > > > > :. > > > Not sure what is in the package, but the last release is 0.9 (I think > package is like 0.7). Which means most of the stuff should work. I would > not like to have svn package, we can always do 0.9.1 if you find nice > stuff which works in svn, but doesn't in 0.9. Yes, that's the ideal solution. > But having 0.9 is still an > improvement :-) Also we need to wait for holger to come back (should be > soon). There is 0.9 in Debian, but as I wrote for example the "py.test -w" isn't working: $ wajig show python-codespeak-lib Package: python-codespeak-lib Priority: optional Section: python Installed-Size: 1680 Maintainer: Guilherme Salgado Architecture: i386 Source: codespeak-lib Version: 0.9.0-3.1 Replaces: python-pylib, python2.3-codespeak-lib, python2.4-codespeak-lib Provides: python2.4-codespeak-lib, python2.5-codespeak-lib Depends: python-central (>= 0.5.8), python (<< 2.6), python (>= 2.4), libc6 (>= 2.6.1-1) Recommends: python-tkinter, python-docutils Suggests: screen, rsync, graphviz, tetex-bin, gs-gpl | gs-esp, ps2eps, subversion Conflicts: python-pylib, python2.3-codespeak-lib, python2.4-codespeak-lib Filename: pool/main/c/codespeak-lib/python-codespeak-lib_0.9.0-3.1_i386.deb Size: 352534 MD5sum: c26760c4b5faf909755b00039206f083 SHA1: d3068c24477aa9b923decdc86d815f960355628b SHA256: ff742dd60a6de29ef2aa80473f16102c932a23f1d9d0da517862853a318147fa Description: The pylib library containing py.test, greenlets and other niceties It includes py.test, whose focus is to get a test environment that's easier to use than the existing ones, py.xml ("a fast'n'easy way to generate xml/html documents"), py.magic.greenlet ("Lightweight in-process concurrent programming") and many more features. . homepage: http://codespeak.net/py/ From oliphant.travis at ieee.org Sun Oct 21 05:25:38 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sat, 20 Oct 2007 22:25:38 -0500 Subject: [py-dev] Help compiling trunk on Windows XP Message-ID: I'm new to compiling Python itself on Windows XP. Can anyone offer advice on how to do it? Is it possible to build the Python trunk with Visual Studio 2005 Express (the free version)? My real purpose is to be able to test patches that are available on Windows so that they can be accepted. There is a patch for allowing mmap to accept the offset parameter that is an important addition that I'd like to help get into Python 2.6 (and Python 3.0). Thanks for any advice, -Travis Oliphant From fijal at genesilico.pl Sun Oct 21 10:29:49 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 21 Oct 2007 10:29:49 +0200 Subject: [py-dev] Help compiling trunk on Windows XP In-Reply-To: References: Message-ID: <471B0DFD.5030609@genesilico.pl> Travis Oliphant wrote: > I'm new to compiling Python itself on Windows XP. Can anyone offer > advice on how to do it? > > Is it possible to build the Python trunk with Visual Studio 2005 Express > (the free version)? > > My real purpose is to be able to test patches that are available on > Windows so that they can be accepted. > > There is a patch for allowing mmap to accept the offset parameter that > is an important addition that I'd like to help get into Python 2.6 (and > Python 3.0). > > Thanks for any advice, > > -Travis Oliphant > > > Hi Travis. This is pylib and py.test mailing list, not the python itself. Seek help here: http://mail.python.org/mailman/listinfo/python-list I cannot really help you :-) Cheers, fijal :.