From yaroslavvb at gmail.com Thu Dec 2 20:14:45 2004 From: yaroslavvb at gmail.com (Yaroslav Bulatov) Date: Thu, 2 Dec 2004 11:14:45 -0800 Subject: [py-dev] Problems with DOS files Message-ID: py.test runs into problems with files using DOS linebreaks (ie \r\n) under Cygwin When I run py.test on a file with DOS linebreaks I get something like def test_5(): ^ SyntaxError: invalid syntax On other hand, when I run the same file using the same python executable, it reports no error. $ /usr/bin/python2.3 test_maxent_classifier.py Traceback (most recent call last): File "test_maxent_classifier.py", line 4, in ? if __name__=='__main__': test_5() File "test_maxent_classifier.py", line 2, in test_5 assert 1==2 AssertionError $ py.test============================= test process starts ============================= testing-mode: inprocess executable : /usr/bin/python2.3 (2.3.4-final-0) =============================================================================== C.. _______________________________________________________________________________ ________________________________ Collect Error ________________________________ def getpycodeobj(self): dotpy = self.check(ext='.py') if dotpy: my_magic = py.std.imp.get_magic() my_timestamp = int(self.mtime()) if __debug__: pycfile = self + 'c' else: pycfile = self + 'o' try: f = pycfile.open('rb') try: header = f.read(8) if len(header) == 8: magic, timestamp = py.std.struct.unpack('<4si', header) if magic == my_magic and timestamp == my_timestamp: return py.std.marshal.load(f) finally: f.close() except IOError: pass s = self.read() > codeobj = compile(s, str(self), 'exec') > /cygdrive/c/Python23/Lib/site-packages/dist-py/py/path/local/local.py, line 36 0 File "/cygdrive/g/research/maxent/classifier/test_maxent_classifier.py", line 1 def test_5(): ^ SyntaxError: invalid syntax ================== tests finished: 2 passed in 0.05 seconds ================== On other hand, everything works fine if I convert the files to unix mode From pobrien at eclypse.org Thu Dec 2 21:08:55 2004 From: pobrien at eclypse.org (Patrick K. O'Brien) Date: Thu, 2 Dec 2004 14:08:55 -0600 Subject: [py-dev] Problems with DOS files In-Reply-To: Message-ID: <200412022009.iB2K9B5M027959@host13.apollohosting.com> The same problem happens on Windows, because the files are being read as binary, maintaining the \r\n-style line endings that mess up the compiler. I'm not sure that I completely understand the py.path code, but I fixed the problem in the following way. I removed the read() method from the py.path.common.FSPathBase class. Then I added read() methods to py.path.local.posix and py.path.local.win, only the win version looks like this, with self.open('r') instead of self.open('rb'): class WinMixin: def read(self): """ read and return a bytestring from reading the path. """ f = self.open('r') try: return f.read() finally: f.close() Also, the py.test file doesn't work as a binary under windows, so I create a batch file, named py.test.bat: @echo off python C:\Code\py\bin\py.test %1 %2 %3 %4 %5 %6 %7 %8 %9 Now I can run "py.test.bat test_whatever.py" under Windows and everything seems to work just fine. It would be great if similar fixes could be checked into svn. And, btw, I'm really enjoying using py.test. Great job, everyone. -- Patrick K. O'Brien > -----Original Message----- > From: py-dev-bounces at codespeak.net [mailto:py-dev-bounces at codespeak.net] > On Behalf Of Yaroslav Bulatov > Sent: Thursday, December 02, 2004 1:15 PM > To: py-dev at codespeak.net > Subject: [py-dev] Problems with DOS files > > py.test runs into problems with files using DOS linebreaks (ie \r\n) > under Cygwin > > When I run py.test on a file with DOS linebreaks I get something like > > def test_5(): > ^ > SyntaxError: invalid syntax From hpk at trillke.net Thu Dec 2 21:51:55 2004 From: hpk at trillke.net (holger krekel) Date: Thu, 2 Dec 2004 21:51:55 +0100 Subject: [py-dev] Problems with DOS files In-Reply-To: <200412022009.iB2K9B5M027959@host13.apollohosting.com> References: <200412022009.iB2K9B5M027959@host13.apollohosting.com> Message-ID: <20041202205154.GQ4550@solar.trillke.net> Hi Patrick, Hi Yaroslav, [Patrick K. O'Brien Thu, Dec 02, 2004 at 02:08:55PM -0600] > The same problem happens on Windows, because the files are being read as > binary, maintaining the \r\n-style line endings that mess up the compiler. > > I'm not sure that I completely understand the py.path code, but I fixed the > problem in the following way. I removed the read() method from the > py.path.common.FSPathBase class. Then I added read() methods to > py.path.local.posix and py.path.local.win, only the win version looks like > this, with self.open('r') instead of self.open('rb'): > > class WinMixin: > > def read(self): > """ read and return a bytestring from reading the path. """ > f = self.open('r') > try: > return f.read() > finally: > f.close() Indeed. I just checked it a fix along those lines. Thanks to Yaruslav for reporting! read() now accepts a mode parameter and you can use 'u' or 'U' for universal newline support, which basically gets turned to 'r' if you run with Python 2.2. I fixed the according "import" problem. Please see if it works for you. > Also, the py.test file doesn't work as a binary under windows, so I create a > batch file, named py.test.bat: > > @echo off > python C:\Code\py\bin\py.test %1 %2 %3 %4 %5 %6 %7 %8 %9 makes sense. > Now I can run "py.test.bat test_whatever.py" under Windows and everything > seems to work just fine. It would be great if similar fixes could be > checked into svn. Hum, i am not very used to windows anymore but i have been worrying about this some time now. What is the simplest/best way to make a script like 'py.test' generally available on windows from a subversion checkout? Your small recipe works for your setup but obviously wouldn't for other users. Hum. On Unix you can "source" the output of 'python .../py/env.py' into your current shell and you will get appropriate access to the py.test script as well as the appropriate directories added to PYTHONPATH. Is there anything similarly simple that can be done under windows? > And, btw, I'm really enjoying using py.test. Great job, everyone. Thanks! Hum, i just took another quick look at your web pages (and btw, i am still interested in PyPerSyst and we did mention it in our PyPy EU proposal :-) and found that you are also doing 'Py' but boy was i relieved to see your actual namespace is 'wx.py' :-) cheers, holger From ianb at colorstudy.com Thu Dec 2 22:31:30 2004 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 02 Dec 2004 15:31:30 -0600 Subject: [py-dev] Problems with DOS files In-Reply-To: <200412022009.iB2K9B5M027959@host13.apollohosting.com> References: <200412022009.iB2K9B5M027959@host13.apollohosting.com> Message-ID: <41AF89B2.2010405@colorstudy.com> Patrick K. O'Brien wrote: > The same problem happens on Windows, because the files are being read as > binary, maintaining the \r\n-style line endings that mess up the compiler. It occurs to me that there might be another bug in there, that py.path might not do the unicode detection that Python normally does (PEP 263; e.g., # -*- encoding: UTF-8 -*-). I haven't tested it (I've never actually used unicode source), but I thought I'd note it since it came to mind. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From pobrien at eclypse.org Thu Dec 2 22:40:09 2004 From: pobrien at eclypse.org (Patrick K. O'Brien) Date: Thu, 2 Dec 2004 15:40:09 -0600 Subject: [py-dev] Problems with DOS files In-Reply-To: <20041202205154.GQ4550@solar.trillke.net> Message-ID: <200412022140.iB2LeQ5M003024@host13.apollohosting.com> > Indeed. I just checked it a fix along those lines. Thanks to > Yaruslav for reporting! > > read() now accepts a mode parameter and you can use 'u' or 'U' for > universal newline support, which basically gets turned to 'r' > if you run with Python 2.2. I fixed the according "import" > problem. Please see if it works for you. Works great. Thanks. > Hum, i am not very used to windows anymore but i have been > worrying about this some time now. What is the simplest/best way > to make a script like 'py.test' generally available on windows > from a subversion checkout? > > Your small recipe works for your setup but obviously wouldn't > for other users. Hum. On Unix you can "source" the output of > 'python .../py/env.py' into your current shell and you will > get appropriate access to the py.test script as well as > the appropriate directories added to PYTHONPATH. Is there > anything similarly simple that can be done under windows? Not that I'm aware of. :-( Take that back. Courtesy of Matthew Scott, here is a clever, two-line solution (concatenate that second line if it breaks): pytest.bat ========== @echo off python -c "import py; from py.__impl__.test.cmdline import main; main()" %1 %2 %3 %4 %5 %6 %7 %8 %9 Useage ====== Add the py\bin directory to your PATH. >From any directory, use pytest.bat the way you would use py.test under Linux. For example: pytest test_whatever.py Note: The file is named pytest.bat, not py.test.bat - the latter confuses Windows if you don't specify the .bat extension when running the file, because it prefers the py.test file but doesn't know what to do with it. > Hum, i just took another quick look at your web pages (and btw, i am still > interested in PyPerSyst and we did mention it in our PyPy EU proposal :-) Matt and I may submit a presentation proposal for the next PyCon, since we both use Pypersyst on a daily basis. Pypersyst has come a long, long way. I think it will blow people's minds once we release the new stuff we have. You'll never want to use any other database again, except as a backend to Pypersyst (well, as a backend to Schevo actually, but that's part of the surprise). If the EU wants to be competitive, they should use Schevo! ;-) Unfortunately the most recent code is no longer on SourceForge, since we wanted to use Subversion. So we moved it all to a private server a while back. But we'll be going public with the latest and greatest in a few weeks. Keep an eye out for Schevo and Pypersyst! :-) > and found that you are also doing 'Py' but boy was i relieved to see your > actual namespace is 'wx.py' :-) Yeah, that was just me getting the "pie" puns out of my system (PyCrust, PyAlaMode, etc.). What's your excuse? (just kidding...) Patrick K. O'Brien From florian.proff.schulze at gmx.net Thu Dec 2 21:27:21 2004 From: florian.proff.schulze at gmx.net (Florian Schulze) Date: Thu, 02 Dec 2004 21:27:21 +0100 Subject: [py-dev] Problems with DOS files In-Reply-To: References: Message-ID: On Thu, 2 Dec 2004 11:14:45 -0800, Yaroslav Bulatov wrote: > py.test runs into problems with files using DOS linebreaks (ie \r\n) > under Cygwin > > When I run py.test on a file with DOS linebreaks I get something like > > def test_5(): > ^ > SyntaxError: invalid syntax I fixed it in py/path/local/local.py by using universal newlines. Does py have to run with python 2.2, if not then the attached patch should work. Regards, Florian Schulze -------------- next part -------------- A non-text attachment was scrubbed... Name: local.py.patch Type: application/octet-stream Size: 464 bytes Desc: not available URL: From briandorsey at gmail.com Fri Dec 3 00:58:12 2004 From: briandorsey at gmail.com (Brian Dorsey) Date: Thu, 2 Dec 2004 15:58:12 -0800 Subject: [py-dev] Problems with DOS files In-Reply-To: <200412022140.iB2LeQ5M003024@host13.apollohosting.com> References: <20041202205154.GQ4550@solar.trillke.net> <200412022140.iB2LeQ5M003024@host13.apollohosting.com> Message-ID: <66e877b704120215582d52de5@mail.gmail.com> On Thu, 2 Dec 2004 15:40:09 -0600, Patrick K. O'Brien wrote: > Take that back. Courtesy of Matthew Scott, here is a clever, two-line > solution (concatenate that second line if it breaks): > > pytest.bat > ========== > @echo off > python -c "import py; from py.__impl__.test.cmdline import main; main()" %1 > %2 %3 %4 %5 %6 %7 %8 %9 I think something along these lines is a good general solution for windows users. A distutils installer could also copy this to the pythonXX\scripts directory. Personally, though... I find it convenient to have the .py extension associated with Python, so in this case, I'm a bit tempted to just copy py.test to the rather oddly named py.test.py. When I do that on my machine, it seems to work, though it doesn't really feel right somehow: C:\Data\BrianDocuments\home\local\py\py\bin>py.test.py ..\test inserting into sys.path: C:\Data\BrianDocuments\home\local\py ============================= test process starts ============================= testing-mode: inprocess executable : C:\Python23\python.exe (2.3.4-final-0) =============================================================================== .............................................. ================== tests finished: 46 passed in 1.28 seconds ================== Take care, -Brian From hpk at trillke.net Fri Dec 3 02:36:43 2004 From: hpk at trillke.net (holger krekel) Date: Fri, 3 Dec 2004 02:36:43 +0100 Subject: [py-dev] Problems with DOS files In-Reply-To: <66e877b704120215582d52de5@mail.gmail.com> References: <20041202205154.GQ4550@solar.trillke.net> <200412022140.iB2LeQ5M003024@host13.apollohosting.com> <66e877b704120215582d52de5@mail.gmail.com> Message-ID: <20041203013643.GT4550@solar.trillke.net> [Brian Dorsey Thu, Dec 02, 2004 at 03:58:12PM -0800] > On Thu, 2 Dec 2004 15:40:09 -0600, Patrick K. O'Brien > wrote: > > > Take that back. Courtesy of Matthew Scott, here is a clever, two-line > > solution (concatenate that second line if it breaks): > > > > pytest.bat > > ========== > > @echo off > > python -c "import py; from py.__impl__.test.cmdline import main; main()" %1 > > %2 %3 %4 %5 %6 %7 %8 %9 > > I think something along these lines is a good general solution for > windows users. A distutils installer could also copy this to the > pythonXX\scripts directory. > > Personally, though... I find it convenient to have the .py extension > associated with Python, so in this case, I'm a bit tempted to just > copy py.test to the rather oddly named py.test.py. Is *.py normally executable on windows after you installed Python? cheers, holger From bob at redivi.com Fri Dec 3 03:08:48 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu, 2 Dec 2004 21:08:48 -0500 Subject: [py-dev] Problems with DOS files In-Reply-To: <20041203013643.GT4550@solar.trillke.net> References: <20041202205154.GQ4550@solar.trillke.net> <200412022140.iB2LeQ5M003024@host13.apollohosting.com> <66e877b704120215582d52de5@mail.gmail.com> <20041203013643.GT4550@solar.trillke.net> Message-ID: <44B10C94-44D0-11D9-9C04-000A9567635C@redivi.com> On Dec 2, 2004, at 8:36 PM, holger krekel wrote: > [Brian Dorsey Thu, Dec 02, 2004 at 03:58:12PM -0800] >> On Thu, 2 Dec 2004 15:40:09 -0600, Patrick K. O'Brien >> wrote: >> >>> Take that back. Courtesy of Matthew Scott, here is a clever, >>> two-line >>> solution (concatenate that second line if it breaks): >>> >>> pytest.bat >>> ========== >>> @echo off >>> python -c "import py; from py.__impl__.test.cmdline import main; >>> main()" %1 >>> %2 %3 %4 %5 %6 %7 %8 %9 >> >> I think something along these lines is a good general solution for >> windows users. A distutils installer could also copy this to the >> pythonXX\scripts directory. >> >> Personally, though... I find it convenient to have the .py extension >> associated with Python, so in this case, I'm a bit tempted to just >> copy py.test to the rather oddly named py.test.py. > > Is *.py normally executable on windows after you installed Python? Yes. Windows doesn't have an executable bit, any extension that is registered by an application as a document type is "executable". If you type the name of a html file into the shell, it will probably open in Internet Explorer. -bob From hpk at trillke.net Fri Dec 3 03:32:59 2004 From: hpk at trillke.net ('holger krekel') Date: Fri, 3 Dec 2004 03:32:59 +0100 Subject: [py-dev] Problems with DOS files In-Reply-To: <200412022140.iB2LeQ5M003024@host13.apollohosting.com> References: <20041202205154.GQ4550@solar.trillke.net> <200412022140.iB2LeQ5M003024@host13.apollohosting.com> Message-ID: <20041203023259.GU4550@solar.trillke.net> [Patrick K. O'Brien Thu, Dec 02, 2004 at 03:40:09PM -0600] > > Hum, i am not very used to windows anymore but i have been > > worrying about this some time now. What is the simplest/best way > > to make a script like 'py.test' generally available on windows > > from a subversion checkout? > > > > Your small recipe works for your setup but obviously wouldn't > > for other users. Hum. On Unix you can "source" the output of > > 'python .../py/env.py' into your current shell and you will > > get appropriate access to the py.test script as well as > > the appropriate directories added to PYTHONPATH. Is there > > anything similarly simple that can be done under windows? > > Not that I'm aware of. :-( > > Take that back. Courtesy of Matthew Scott, here is a clever, two-line > solution (concatenate that second line if it breaks): > > pytest.bat > ========== > @echo off > python -c "import py; from py.__impl__.test.cmdline import main; main()" %1 > %2 %3 %4 %5 %6 %7 %8 %9 Interesting. Though this still seems to depend on A) having 'py' in your PYTHONPATH B) always wanting the same version of 'py' to be used by py.test But on Unix the python-hashbanged script py/bin/py.test uses the logic in 'py/bin/_findpy.py' to determine a "nearby" version of the py lib. This is useful when you have multiple checkouts, say versions in the form of svn-externals, and want to fix bugs in your local "py" version while working on some tests of your project at hand. (Pretty common use case with py.test if you ask me :-). So i wonder how/if we could remedy the above A) and B) to get an equivalent windows version. > > Hum, i just took another quick look at your web pages (and btw, i am still > > interested in PyPerSyst and we did mention it in our PyPy EU proposal :-) > > Matt and I may submit a presentation proposal for the next PyCon, since we > both use Pypersyst on a daily basis. Pypersyst has come a long, long way. > I think it will blow people's minds once we release the new stuff we have. > You'll never want to use any other database again, except as a backend to > Pypersyst (well, as a backend to Schevo actually, but that's part of the > surprise). If the EU wants to be competitive, they should use Schevo! ;-) Hehe, that sounds good! Now i only desparately need a need for a database :-) > Unfortunately the most recent code is no longer on SourceForge, since we > wanted to use Subversion. So we moved it all to a private server a while > back. But we'll be going public with the latest and greatest in a few > weeks. Keep an eye out for Schevo and Pypersyst! :-) Of course, you are invited to move your project to codespeak if you like. We have some 2 1/2 years worth of subversion experience and are soon to install a new setup with a 'trac' environment and a pretty reliable machine ... we hope :-) > > and found that you are also doing 'Py' but boy was i relieved to see your > > actual namespace is 'wx.py' :-) > > Yeah, that was just me getting the "pie" puns out of my system (PyCrust, > PyAlaMode, etc.). What's your excuse? (just kidding...) Hehe, i bite on this one. There are basically three reasons why it's named 'py' now: - I could convince Armin in a weak moment :) - 'py.test' seems to make "idiomatic" sense and resembles some of the ambition to make it a generally useful test tool for all kinds of python projects. - py. should give nice completion also for any upcoming scripts provided by the py lib. - The former 'std' seemed too broad and sounded more static than 'py' wants to be. You know, py.test and its underlying library are still part of the quest for the holy grail .. i hear the sound of coconuts ... monty-pythonic greetings, holger From hpk at trillke.net Fri Dec 3 03:34:22 2004 From: hpk at trillke.net (holger krekel) Date: Fri, 3 Dec 2004 03:34:22 +0100 Subject: [py-dev] Problems with DOS files In-Reply-To: <44B10C94-44D0-11D9-9C04-000A9567635C@redivi.com> References: <20041202205154.GQ4550@solar.trillke.net> <200412022140.iB2LeQ5M003024@host13.apollohosting.com> <66e877b704120215582d52de5@mail.gmail.com> <20041203013643.GT4550@solar.trillke.net> <44B10C94-44D0-11D9-9C04-000A9567635C@redivi.com> Message-ID: <20041203023422.GV4550@solar.trillke.net> > >Is *.py normally executable on windows after you installed Python? > > Yes. Windows doesn't have an executable bit, any extension that is > registered by an application as a document type is "executable". If > you type the name of a html file into the shell, it will probably open > in Internet Explorer. Ok, let me refine the question: does a default python install register this mapping and/or do developers usually have this mapping active? holger From bob at redivi.com Fri Dec 3 03:37:14 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu, 2 Dec 2004 21:37:14 -0500 Subject: [py-dev] Problems with DOS files In-Reply-To: <20041203023422.GV4550@solar.trillke.net> References: <20041202205154.GQ4550@solar.trillke.net> <200412022140.iB2LeQ5M003024@host13.apollohosting.com> <66e877b704120215582d52de5@mail.gmail.com> <20041203013643.GT4550@solar.trillke.net> <44B10C94-44D0-11D9-9C04-000A9567635C@redivi.com> <20041203023422.GV4550@solar.trillke.net> Message-ID: <3DFA59F8-44D4-11D9-9C04-000A9567635C@redivi.com> On Dec 2, 2004, at 9:34 PM, holger krekel wrote: >>> Is *.py normally executable on windows after you installed Python? >> >> Yes. Windows doesn't have an executable bit, any extension that is >> registered by an application as a document type is "executable". If >> you type the name of a html file into the shell, it will probably open >> in Internet Explorer. > > Ok, let me refine the question: does a default python install register > this mapping and/or do developers usually have this mapping active? That's what I meant by "Yes." :) -bob From hpk at trillke.net Fri Dec 3 03:48:21 2004 From: hpk at trillke.net (holger krekel) Date: Fri, 3 Dec 2004 03:48:21 +0100 Subject: [py-dev] Problems with DOS files In-Reply-To: References: Message-ID: <20041203024821.GW4550@solar.trillke.net> [Florian Schulze Thu, Dec 02, 2004 at 09:27:21PM +0100] > On Thu, 2 Dec 2004 11:14:45 -0800, Yaroslav Bulatov > wrote: > > >py.test runs into problems with files using DOS linebreaks (ie \r\n) > >under Cygwin > > > >When I run py.test on a file with DOS linebreaks I get something like > > > > def test_5(): > > ^ > >SyntaxError: invalid syntax > > I fixed it in py/path/local/local.py by using universal newlines. Does py > have to run with python 2.2, if not then the attached patch should work. right, thanks. I did a similar fix in a slightly more general way: read() accepts a mode parameter itself. cheers, holger From pobrien at eclypse.org Fri Dec 3 15:37:22 2004 From: pobrien at eclypse.org (Patrick K. O'Brien) Date: Fri, 3 Dec 2004 08:37:22 -0600 Subject: [py-dev] Problems with DOS files In-Reply-To: <3DFA59F8-44D4-11D9-9C04-000A9567635C@redivi.com> Message-ID: <200412031437.iB3EbeCG030827@host13.apollohosting.com> > On Behalf Of Bob Ippolito > > On Dec 2, 2004, at 9:34 PM, holger krekel wrote: > > >>> Is *.py normally executable on windows after you installed Python? > >> > >> Yes. Windows doesn't have an executable bit, any extension that is > >> registered by an application as a document type is "executable". If > >> you type the name of a html file into the shell, it will probably open > >> in Internet Explorer. > > > > Ok, let me refine the question: does a default python install register > > this mapping and/or do developers usually have this mapping active? > > That's what I meant by "Yes." :) Well, unfortunately there are still some issues. On Windows you map to some default behavior, which may or may not be to execute the file. On my machine, for example, I've changed the default behavior to load the .py file into Emacs. So this solution wouldn't work for me. :-( Patrick K. O'Brien From faassen at infrae.com Fri Dec 3 18:10:51 2004 From: faassen at infrae.com (Martijn Faassen) Date: Fri, 03 Dec 2004 18:10:51 +0100 Subject: [py-dev] Problems with DOS files In-Reply-To: <20041203023259.GU4550@solar.trillke.net> References: <20041202205154.GQ4550@solar.trillke.net> <200412022140.iB2LeQ5M003024@host13.apollohosting.com> <20041203023259.GU4550@solar.trillke.net> Message-ID: <41B09E1B.8020106@infrae.com> 'holger krekel' wrote: > [Patrick K. O'Brien Thu, Dec 02, 2004 at 03:40:09PM -0600] [snip] >>Matt and I may submit a presentation proposal for the next PyCon, since we >>both use Pypersyst on a daily basis. Pypersyst has come a long, long way. >>I think it will blow people's minds once we release the new stuff we have. >>You'll never want to use any other database again, except as a backend to >>Pypersyst (well, as a backend to Schevo actually, but that's part of the >>surprise). If the EU wants to be competitive, they should use Schevo! ;-) > > Hehe, that sounds good! Now i only desparately need a need for a database :-) Yeah, this blurb makes me want to play with it too. Infrae luckily does need databases, so this sounds quite interesting. >>Unfortunately the most recent code is no longer on SourceForge, since we >>wanted to use Subversion. So we moved it all to a private server a while >>back. But we'll be going public with the latest and greatest in a few >>weeks. Keep an eye out for Schevo and Pypersyst! :-) > > Of course, you are invited to move your project to codespeak if you like. > We have some 2 1/2 years worth of subversion experience and are > soon to install a new setup with a 'trac' environment and a pretty > reliable machine ... we hope :-) As a satisfied developer on codespeak I can only recommend it. :) Regards, Martijn From hpk at trillke.net Sat Dec 4 10:11:01 2004 From: hpk at trillke.net ('holger krekel') Date: Sat, 4 Dec 2004 10:11:01 +0100 Subject: [py-dev] Problems with DOS files In-Reply-To: <200412031437.iB3EbeCG030827@host13.apollohosting.com> References: <3DFA59F8-44D4-11D9-9C04-000A9567635C@redivi.com> <200412031437.iB3EbeCG030827@host13.apollohosting.com> Message-ID: <20041204091101.GL4550@solar.trillke.net> [Patrick K. O'Brien Fri, Dec 03, 2004 at 08:37:22AM -0600] > > On Behalf Of Bob Ippolito > > > > On Dec 2, 2004, at 9:34 PM, holger krekel wrote: > > > > >>> Is *.py normally executable on windows after you installed Python? > > >> > > >> Yes. Windows doesn't have an executable bit, any extension that is > > >> registered by an application as a document type is "executable". If > > >> you type the name of a html file into the shell, it will probably open > > >> in Internet Explorer. > > > > > > Ok, let me refine the question: does a default python install register > > > this mapping and/or do developers usually have this mapping active? > > > > That's what I meant by "Yes." :) > > Well, unfortunately there are still some issues. On Windows you map to some > default behavior, which may or may not be to execute the file. On my > machine, for example, I've changed the default behavior to load the .py file > into Emacs. So this solution wouldn't work for me. :-( OK, so my current idea is to create src/winscript/ and src/unixscript/ directories and put scripts in there. With appropriate "py.test.bat" and "py.test" scripts respectively. Both scripts should contain enough logic to detect a nearby "py" version or just fall back to assume the user set PYTHONPATH accordingly. IOW, they would work if you have a "local" svn-external'ed version of the py lib for some project. Does that make sense? If so, the only open question is how to make the .bat windows script do the "nearby" detection. If the .bat language is really too limited to do this then there should be a "py.test.py" in winscript/ that has to be copied alongside "py.test.bat" and hopefully it is possible to dynamically get to "py.test.py" from a "py.test.bat" in the same directory?! cheers, holger P.S.: i know that for some of you (especially Bob) this all does not seem to be worth it. But i believe in developer setups that don't require to run distutils but work seemlessly with multiple checkouts appropriately ... After all, how would you keep multiple versions of the py lib managed by distutils? From pobrien at eclypse.org Sun Dec 5 18:53:09 2004 From: pobrien at eclypse.org (Patrick K. O'Brien) Date: Sun, 5 Dec 2004 11:53:09 -0600 Subject: [py-dev] Problems with DOS files In-Reply-To: <41B09E1B.8020106@infrae.com> Message-ID: <200412051753.iB5Hrbo3004294@host13.apollohosting.com> > Yeah, this blurb makes me want to play with it too. Infrae luckily does > need databases, so this sounds quite interesting. Information will be available soon on these sites: Pypersyst: http://www.pypersyst.org Schevo: http://www.schevo.org And we are seriously considering the offer to host the code on the codespeak Subversion repository. -- Patrick K. O'Brien Orbtech From briandorsey at gmail.com Mon Dec 6 18:58:33 2004 From: briandorsey at gmail.com (Brian Dorsey) Date: Mon, 6 Dec 2004 09:58:33 -0800 Subject: [py-dev] Problems with DOS files In-Reply-To: <20041204091101.GL4550@solar.trillke.net> References: <3DFA59F8-44D4-11D9-9C04-000A9567635C@redivi.com> <200412031437.iB3EbeCG030827@host13.apollohosting.com> <20041204091101.GL4550@solar.trillke.net> Message-ID: <66e877b70412060958c128221@mail.gmail.com> On Sat, 4 Dec 2004 10:11:01 +0100, holger krekel wrote: > If so, the only open question is how to make the .bat windows > script do the "nearby" detection. If the .bat language is really > too limited to do this then there should be a "py.test.py" in > winscript/ that has to be copied alongside "py.test.bat" and hopefully > it is possible to dynamically get to "py.test.py" from a > "py.test.bat" in the same directory?! > Holger, I'm not sure if this relates or not, but I ran into it again today, and it might serve as inspiration if nothing else: http://effbot.org/zone/exemaker.htm "ExeMaker is a small tool that takes a Python script, copies it to a program directory, and creates a Windows EXE file in the same directory. When you run that EXE, it automatically runs the script. ... In other words, ExeMaker lets you install command-line programs written in Python along with other utilities, and use them without having to bother with PY associations, BAT files, or other workarounds." Take care, -Brian From p.f.moore at gmail.com Tue Dec 7 12:30:55 2004 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 7 Dec 2004 11:30:55 +0000 Subject: [py-dev] Problems with DOS files In-Reply-To: <66e877b70412060958c128221@mail.gmail.com> References: <3DFA59F8-44D4-11D9-9C04-000A9567635C@redivi.com> <200412031437.iB3EbeCG030827@host13.apollohosting.com> <20041204091101.GL4550@solar.trillke.net> <66e877b70412060958c128221@mail.gmail.com> Message-ID: <79990c6b04120703301622a001@mail.gmail.com> On Mon, 6 Dec 2004 09:58:33 -0800, Brian Dorsey wrote: > I'm not sure if this relates or not, but I ran into it again today, > and it might serve as inspiration if nothing else: > > http://effbot.org/zone/exemaker.htm > > "ExeMaker is a small tool that takes a Python script, copies it to a > program directory, and creates a Windows EXE file in the same > directory. When you run that EXE, it automatically runs the script. > ... > In other words, ExeMaker lets you install command-line programs > written in Python along with other utilities, and use them without > having to bother with PY associations, BAT files, or other > workarounds." One problem with using a BAT file (which is more of an issue with long-running scripts like webservers) is that if you use Ctrl-C to interrupt the script, Windows gives a prompt "Do you want to terminate the batch file? (Y/N/Cancel)" - even if the script execution is the last line of the BAT file. My preference is for just the .py file, as I have suitable associations set up. But I can see this may be an issue for some people. From that POV, exemaker may be a good compromise. Paul. From hpk at trillke.net Tue Dec 7 12:36:14 2004 From: hpk at trillke.net (holger krekel) Date: Tue, 7 Dec 2004 12:36:14 +0100 Subject: [py-dev] Problems with DOS files In-Reply-To: <79990c6b04120703301622a001@mail.gmail.com> References: <3DFA59F8-44D4-11D9-9C04-000A9567635C@redivi.com> <200412031437.iB3EbeCG030827@host13.apollohosting.com> <20041204091101.GL4550@solar.trillke.net> <66e877b70412060958c128221@mail.gmail.com> <79990c6b04120703301622a001@mail.gmail.com> Message-ID: <20041207113614.GE4550@solar.trillke.net> [Paul Moore Tue, Dec 07, 2004 at 11:30:55AM +0000] > On Mon, 6 Dec 2004 09:58:33 -0800, Brian Dorsey wrote: > > I'm not sure if this relates or not, but I ran into it again today, > > and it might serve as inspiration if nothing else: > > > > http://effbot.org/zone/exemaker.htm > > > > "ExeMaker is a small tool that takes a Python script, copies it to a > > program directory, and creates a Windows EXE file in the same > > directory. When you run that EXE, it automatically runs the script. > > ... > > In other words, ExeMaker lets you install command-line programs > > written in Python along with other utilities, and use them without > > having to bother with PY associations, BAT files, or other > > workarounds." > > One problem with using a BAT file (which is more of an issue with > long-running scripts like webservers) is that if you use Ctrl-C to > interrupt the script, Windows gives a prompt "Do you want to terminate > the batch file? (Y/N/Cancel)" - even if the script execution is the > last line of the BAT file. > > My preference is for just the .py file, as I have suitable > associations set up. But I can see this may be an issue for some > people. From that POV, exemaker may be a good compromise. Yes, matches my current thinking. Do you think that distributing a '.exe' directly would already work in many cases? Or is this too platform/python-version dependent to be useful for Windows? I guess i finally need to get myself a Windows PC to know what i am talking about :-) cheers & thanks for the suggestions, holger From lac at strakt.com Tue Dec 7 12:50:24 2004 From: lac at strakt.com (Laura Creighton) Date: Tue, 07 Dec 2004 12:50:24 +0100 Subject: [py-dev] Problems with DOS files In-Reply-To: Message from hpk@trillke.net (holger krekel) of "Tue, 07 Dec 2004 12:36:14 +0100." <20041207113614.GE4550@solar.trillke.net> References: <3DFA59F8-44D4-11D9-9C04-000A9567635C@redivi.com> <200412031437.iB3EbeCG030827@host13.apollohosting.com> <20041204091101.GL4550@solar.trillke.net> <66e877b70412060958c128221@mail.gmail.com> <79990c6b04120703301622a001@mail.gmail.com> <20041207113614.GE4550@solar.trillke.net> Message-ID: <200412071150.iB7BoPlq011772@ratthing-b246.strakt.com> In a message of Tue, 07 Dec 2004 12:36:14 +0100, holger krekel writes: > >I guess i finally need to get myself a Windows PC to know what >i am talking about :-) ah, you can just have 2 separate boot partitions on your laptop, or whatever ... > >cheers & thanks for the suggestions, > > holger Laura From hpk at trillke.net Tue Dec 7 13:05:19 2004 From: hpk at trillke.net (holger krekel) Date: Tue, 7 Dec 2004 13:05:19 +0100 Subject: [py-dev] Problems with DOS files In-Reply-To: <200412071150.iB7BoPlq011772@ratthing-b246.strakt.com> References: <3DFA59F8-44D4-11D9-9C04-000A9567635C@redivi.com> <200412031437.iB3EbeCG030827@host13.apollohosting.com> <20041204091101.GL4550@solar.trillke.net> <66e877b70412060958c128221@mail.gmail.com> <79990c6b04120703301622a001@mail.gmail.com> <20041207113614.GE4550@solar.trillke.net> <200412071150.iB7BoPlq011772@ratthing-b246.strakt.com> Message-ID: <20041207120519.GF4550@solar.trillke.net> [Laura Creighton Tue, Dec 07, 2004 at 12:50:24PM +0100] > In a message of Tue, 07 Dec 2004 12:36:14 +0100, holger krekel writes: > > > >I guess i finally need to get myself a Windows PC to know what > >i am talking about :-) > > ah, you can just have 2 separate boot partitions on your laptop, or whatever ... on my upcoming mac laptop? (i intend to get one because my old intel one just broke ... :-) Well, and anyway i really need a permanent windows machine somewhere because i want to have interactive testing across platform boundaries aka py.test --platform=win32 or some such, anyway :-) Much of the neccessary code is already there if you run "py.test --session" but i need to fix small details to make the py lib fully work on windows first ... cheers, holger From bob at redivi.com Tue Dec 7 17:30:42 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue, 7 Dec 2004 11:30:42 -0500 Subject: [py-dev] Problems with DOS files In-Reply-To: <20041207113614.GE4550@solar.trillke.net> References: <3DFA59F8-44D4-11D9-9C04-000A9567635C@redivi.com> <200412031437.iB3EbeCG030827@host13.apollohosting.com> <20041204091101.GL4550@solar.trillke.net> <66e877b70412060958c128221@mail.gmail.com> <79990c6b04120703301622a001@mail.gmail.com> <20041207113614.GE4550@solar.trillke.net> Message-ID: <567CECC0-486D-11D9-8917-000A9567635C@redivi.com> On Dec 7, 2004, at 6:36 AM, holger krekel wrote: > [Paul Moore Tue, Dec 07, 2004 at 11:30:55AM +0000] >> On Mon, 6 Dec 2004 09:58:33 -0800, Brian Dorsey >> wrote: >>> I'm not sure if this relates or not, but I ran into it again today, >>> and it might serve as inspiration if nothing else: >>> >>> http://effbot.org/zone/exemaker.htm >>> >>> "ExeMaker is a small tool that takes a Python script, copies it to a >>> program directory, and creates a Windows EXE file in the same >>> directory. When you run that EXE, it automatically runs the script. >>> ... >>> In other words, ExeMaker lets you install command-line programs >>> written in Python along with other utilities, and use them without >>> having to bother with PY associations, BAT files, or other >>> workarounds." >> >> One problem with using a BAT file (which is more of an issue with >> long-running scripts like webservers) is that if you use Ctrl-C to >> interrupt the script, Windows gives a prompt "Do you want to terminate >> the batch file? (Y/N/Cancel)" - even if the script execution is the >> last line of the BAT file. >> >> My preference is for just the .py file, as I have suitable >> associations set up. But I can see this may be an issue for some >> people. From that POV, exemaker may be a good compromise. > > Yes, matches my current thinking. Do you think that distributing > a '.exe' directly would already work in many cases? Or is this > too platform/python-version dependent to be useful for Windows? > > I guess i finally need to get myself a Windows PC to know what > i am talking about :-) It's possible to do the same thing I do with py2app, namely, load up the appropriate python runtime dynamically (probably based on some registry key or ini file) and run the script that way.. this makes sure that you don't suffer any ABI problems so long as you are using functions with the same signature. It's ugly, but quite nice because you can precompile this and use it to bootstrap any Python program. Here's what it looks like on Mac OS X (of course all the Objective-C code would go away, and dyld functions would be replaced by equivalent win32 stuff): http://svn.red-bean.com/bob/py2app/trunk/src/py2app/apptemplate/src/ main.m It's also possible to have py.test.exe be written in pure C, which just figures out which your current Python is and spawns it with the py.test script and appropriate argv. -bob From hpk at trillke.net Thu Dec 9 00:49:49 2004 From: hpk at trillke.net (holger krekel) Date: Thu, 9 Dec 2004 00:49:49 +0100 Subject: [py-dev] Re: [py-svn] r7784 - in py/dist/py: . test In-Reply-To: <20041208234417.C50A45A95A@thoth.codespeak.net> References: <20041208234417.C50A45A95A@thoth.codespeak.net> Message-ID: <20041208234949.GP6031@solar.trillke.net> Hi Ian, hi All, i have implemented generative tests: > New Revision: 7784 > implemented support for Generator (collection) functions and methods. > plus a couple of tests. > > With this new hack you can yield parametrized tests in an easy > ("the best API is one that doesn't exist") way: > > def somefunc(x, y): > assert x * y == 42 > > def test_generator(): > yield somefunc, 6, 7 > yield somefunc, 7, 6 However, i am not completly sure yet about appropriate setup/teardown semantics. Should setup_method()/teardown_method() be performed for each run of a generated callable+*args/**kwargs thingie or just for the test generator itself? (test generator = the test function that actually yields callables/args/kwargs tuples). cheers, holger From ianb at colorstudy.com Thu Dec 9 08:37:46 2004 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 09 Dec 2004 01:37:46 -0600 Subject: [py-dev] rootdir and --session Message-ID: <41B800CA.8050000@colorstudy.com> When I run py.test --session I get an error (and perhaps because that's the only option that requires configuration?): Traceback (most recent call last): File "/home/ianb/py/dist/py/bin/py.test", line 5, in ? main() File "/home/ianb/py/dist/py/test/cmdline.py", line 29, in main run.session(args, filenames) File "/home/ianb/py/dist/py/test/run.py", line 174, in session rootdir = py.path.local(py.test.config.getfirst('rootdir')) File "/home/ianb/py/dist/py/test/config.py", line 50, in getfirst raise AttributeError, param AttributeError: rootdir The only place it seems to be looking for configuration is py/test/defaultconfig.py, and it's not clear how I add other files. And should rootdir be a command-line option (default '.')? -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Thu Dec 9 09:24:34 2004 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 09 Dec 2004 02:24:34 -0600 Subject: [py-dev] .py files without trailing newlines Message-ID: <41B80BC2.70008@colorstudy.com> Just noticed that I get a syntax error if I load a Python file (with extpy) that doesn't have a trailing newline. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From hpk at trillke.net Thu Dec 9 09:32:01 2004 From: hpk at trillke.net (holger krekel) Date: Thu, 9 Dec 2004 09:32:01 +0100 Subject: [py-dev] .py files without trailing newlines In-Reply-To: <41B80BC2.70008@colorstudy.com> References: <41B80BC2.70008@colorstudy.com> Message-ID: <20041209083201.GR6031@solar.trillke.net> Hi Ian, [Ian Bicking Thu, Dec 09, 2004 at 02:24:34AM -0600] > Just noticed that I get a syntax error if I load a Python file (with > extpy) that doesn't have a trailing newline. can you modify in py/path/local/local.py in getpycode() about in the middle: s = self.read(mode='rU') + '\n' and see if it helps? Would be great if you can add a small test and the above fix (if it helps) and check it in. cheers, holger P.S.: i also quick-fixed the rootdir-problem. test configuration indeed needs documentation and some refactoring ... From hpk at trillke.net Thu Dec 16 12:00:00 2004 From: hpk at trillke.net (holger krekel) Date: Thu, 16 Dec 2004 12:00:00 +0100 Subject: [py-dev] changes ahead ... Message-ID: <20041216110000.GX23584@solar.trillke.net> hi py-dev, just a short warning note that in the upcoming days there are going to be quite some changes to py.test. There is no explicit plan to break it but you'll never know :-) Basically we are going to try to port PyPy's tests to py.test and expand some of the capabilities of py.test for this. cheers, holger From florian.proff.schulze at gmx.net Thu Dec 16 18:58:54 2004 From: florian.proff.schulze at gmx.net (Florian Schulze) Date: Thu, 16 Dec 2004 18:58:54 +0100 Subject: [py-dev] Greenlets Message-ID: Hi! I just saw that greenlets where imported into py and I wanted to try them out. Unfortunately they don't compile on windows, the offending line is #536: PyObject_HEAD_INIT(&PyType_Type) greenlet.c(536) : error C2099: Initialisierung ist keine Konstante roughly translated: greenlet.c(536) : error C2099: Initializer is not constant I looked into other extension modules and there all those lines read like this: PyObject_HEAD_INIT(NULL) It compiled with that line, but it crashes after the greenlets ran through. It's an really interesting idea though. Regards, Florian Schulze ps: Are Armin Rigo and Christian Tismer on this list? From hpk at trillke.net Fri Dec 17 02:47:55 2004 From: hpk at trillke.net (holger krekel) Date: Fri, 17 Dec 2004 02:47:55 +0100 Subject: [py-dev] Greenlets In-Reply-To: References: Message-ID: <20041217014755.GE23584@solar.trillke.net> [Florian Schulze Thu, Dec 16, 2004 at 06:58:54PM +0100] > Hi! > > I just saw that greenlets where imported into py and I wanted to try them > out. Unfortunately they don't compile on windows, the offending line is > #536: > PyObject_HEAD_INIT(&PyType_Type) > greenlet.c(536) : error C2099: Initialisierung ist keine Konstante > roughly translated: > greenlet.c(536) : error C2099: Initializer is not constant > I looked into other extension modules and there all those lines read like > this: > PyObject_HEAD_INIT(NULL) > > It compiled with that line, but it crashes after the greenlets ran through. Does installing distutils-based C-coded packages usually work for you? > It's an really interesting idea though. Greenlets or integrating C-modules on the fly or both? :-) > ps: Are Armin Rigo and Christian Tismer on this list? Armin yes, but not christian i think. cheers, holger From florian.proff.schulze at gmx.net Fri Dec 17 12:41:02 2004 From: florian.proff.schulze at gmx.net (Florian Schulze) Date: Fri, 17 Dec 2004 12:41:02 +0100 Subject: [py-dev] Greenlets In-Reply-To: <20041217014755.GE23584@solar.trillke.net> References: <20041217014755.GE23584@solar.trillke.net> Message-ID: On Fri, 17 Dec 2004 02:47:55 +0100, holger krekel wrote: > [Florian Schulze Thu, Dec 16, 2004 at 06:58:54PM +0100] >> Hi! >> >> I just saw that greenlets where imported into py and I wanted to try >> them >> out. Unfortunately they don't compile on windows, the offending line is >> #536: >> PyObject_HEAD_INIT(&PyType_Type) >> greenlet.c(536) : error C2099: Initialisierung ist keine Konstante >> roughly translated: >> greenlet.c(536) : error C2099: Initializer is not constant >> I looked into other extension modules and there all those lines read >> like >> this: >> PyObject_HEAD_INIT(NULL) >> >> It compiled with that line, but it crashes after the greenlets ran >> through. > > Does installing distutils-based C-coded packages usually work for you? Yes, if zope3, numarray, pygame etc counts ;) >> It's an really interesting idea though. > > Greenlets or integrating C-modules on the fly or both? :-) I only tried Greenlets yet, but the on the fly C-module stuff is also very cool. What happens when I start it with Python 2.3 it compiles and then I start it with Python 2.4, does it detect that it has to recompile? >> ps: Are Armin Rigo and Christian Tismer on this list? > > Armin yes, but not christian i think. ok. > cheers, > > holger Regards, Florian Schulze From hpk at trillke.net Fri Dec 17 17:47:07 2004 From: hpk at trillke.net (holger krekel) Date: Fri, 17 Dec 2004 17:47:07 +0100 Subject: [py-dev] Greenlets In-Reply-To: References: <20041217014755.GE23584@solar.trillke.net> Message-ID: <20041217164707.GL15414@solar.trillke.net> On Fri, Dec 17, 2004 at 12:41 +0100, Florian Schulze wrote: > On Fri, 17 Dec 2004 02:47:55 +0100, holger krekel wrote: > >[Florian Schulze Thu, Dec 16, 2004 at 06:58:54PM +0100] > >>I just saw that greenlets where imported into py and I wanted to try > >>them > >>out. Unfortunately they don't compile on windows, the offending line is > >>#536: > >>PyObject_HEAD_INIT(&PyType_Type) > >>greenlet.c(536) : error C2099: Initialisierung ist keine Konstante > >>roughly translated: > >>greenlet.c(536) : error C2099: Initializer is not constant > >>I looked into other extension modules and there all those lines read > >>like > >>this: > >>PyObject_HEAD_INIT(NULL) > >> > >>It compiled with that line, but it crashes after the greenlets ran > >>through. > > > >Does installing distutils-based C-coded packages usually work for you? > > Yes, if zope3, numarray, pygame etc counts ;) OK, Armin worked around an underlying compiler limitation so you might retry by using an updated py trunk and doing python -c "import py ; py.magic.greenlet()" this works fine on my linux machine. Look into py/__init__.py if you want to find out where "magic.greenlet" comes from ... > >>It's an really interesting idea though. > > > >Greenlets or integrating C-modules on the fly or both? :-) > > I only tried Greenlets yet, but the on the fly C-module stuff is also very > cool. What happens when I start it with Python 2.3 it compiles and then I > start it with Python 2.4, does it detect that it has to recompile? Actually, it should detect this (but i haven't tried). Eventually, i'd like to provide transparent means to store so/pycs files into separated locations so you only compile-on-the-fly once per python binary/platform or if the file changes ... The idea here is to have a single checkout for multiple python versions in an efficient way on windows/linux/unix platforms. cheers, holger From lac at strakt.com Sun Dec 19 17:06:47 2004 From: lac at strakt.com (Laura Creighton) Date: Sun, 19 Dec 2004 17:06:47 +0100 Subject: [py-dev] move tests into their own directory ... Message-ID: <200412191606.iBJG6l37028773@ratthing-b246.strakt.com> For what it's worth, I think it is a very good idea. Laura From hpk at trillke.net Sun Dec 19 17:15:38 2004 From: hpk at trillke.net (holger krekel) Date: Sun, 19 Dec 2004 17:15:38 +0100 Subject: [py-dev] move tests into their own directory ... In-Reply-To: <200412191606.iBJG6l37028773@ratthing-b246.strakt.com> References: <200412191606.iBJG6l37028773@ratthing-b246.strakt.com> Message-ID: <20041219161537.GD16238@solar.trillke.net> Hi Laura, On Sun, Dec 19, 2004 at 17:06 +0100, Laura Creighton wrote: > For what it's worth, I think it is a very good idea. thanks for your attention and your comment! I was so far in doubts whether to keep the tests close to the implementation in the same directory. But i now lean towards using a separate directory. (but py.test will continue to find all test_*.py and *_test.py files, of course). Eventually i want to get rid of *.pyc files as well in favour of installing them in a platform specific directory. The basic idea here is to have completely clean implementation directories and being able to use the same py-lib checkout for multiple python versions effectively (without recompiling all the time). This need became more apparent, now that we support C-translation on the fly (e.g. py.magic.greenlet). However, it's not completly obvious how to get to such persistent storage and due to permission issues it probably needs to be per-user as well (which should hardly be a problem). cheers & greetings from R'dam, holger P.S.: btw, we are happily using your utestconvert script for converting pypy's unittest.py based tests and so far it works well! I plan to make it available one way or another with the py lib if you don't mind. From lac at strakt.com Sun Dec 19 17:32:55 2004 From: lac at strakt.com (Laura Creighton) Date: Sun, 19 Dec 2004 17:32:55 +0100 Subject: [py-dev] move tests into their own directory ... In-Reply-To: Message from hpk@trillke.net (holger krekel) of "Sun, 19 Dec 2004 17:15:38 +0100." <20041219161537.GD16238@solar.trillke.net> References: <200412191606.iBJG6l37028773@ratthing-b246.strakt.com> <20041219161537.GD16238@solar.trillke.net> Message-ID: <200412191632.iBJGWtSX028867@ratthing-b246.strakt.com> In a message of Sun, 19 Dec 2004 17:15:38 +0100, holger krekel writes: >Hi Laura, > >On Sun, Dec 19, 2004 at 17:06 +0100, Laura Creighton wrote: >> For what it's worth, I think it is a very good idea. > >thanks for your attention and your comment! I was >so far in doubts whether to keep the tests close to >the implementation in the same directory. But i now lean >towards using a separate directory. (but py.test >will continue to find all test_*.py and *_test.py files, >of course). > >Eventually i want to get rid of *.pyc files as well in >favour of installing them in a platform specific directory. >The basic idea here is to have completely clean >implementation directories and being able to use >the same py-lib checkout for multiple python versions >effectively (without recompiling all the time). This need >became more apparent, now that we support C-translation >on the fly (e.g. py.magic.greenlet). This should prove to be extremely popular. The Holger Krekel fan club may increase. >However, it's not completly obvious how to get to such >persistent storage and due to permission issues it probably >needs to be per-user as well (which should hardly be a >problem). I don't know how windows is handling users these days. In the old days that would be a problem, since there weren't any users. Do we need to support this, I wonder out loud ... > >cheers & greetings from R'dam, > > holger > >P.S.: btw, we are happily using your utestconvert script > for converting pypy's unittest.py based tests and > so far it works well! I plan to make it available > one way or another with the py lib if you don't mind. This is great to hear. Of course I don't mind -- I thought that this was part of the plan already. But thank you for asking. By the way, in addition to commenting back in 'assertRaises becomes raises' there is one other defect of my program. It does not remove the 'import unittest' lines. Or have you already added that while I was not looking? cheers from G?teborg, Laura From hpk at trillke.net Sun Dec 19 17:39:30 2004 From: hpk at trillke.net (holger krekel) Date: Sun, 19 Dec 2004 17:39:30 +0100 Subject: [py-dev] move tests into their own directory ... In-Reply-To: <200412191632.iBJGWtSX028867@ratthing-b246.strakt.com> References: <200412191606.iBJG6l37028773@ratthing-b246.strakt.com> <20041219161537.GD16238@solar.trillke.net> <200412191632.iBJGWtSX028867@ratthing-b246.strakt.com> Message-ID: <20041219163930.GE16238@solar.trillke.net> On Sun, Dec 19, 2004 at 17:32 +0100, Laura Creighton wrote: > In a message of Sun, 19 Dec 2004 17:15:38 +0100, holger krekel writes: > >P.S.: btw, we are happily using your utestconvert script > > for converting pypy's unittest.py based tests and > > so far it works well! I plan to make it available > > one way or another with the py lib if you don't mind. > > This is great to hear. Of course I don't mind -- I thought that > this was part of the plan already. But thank you for asking. > By the way, in addition to commenting back in 'assertRaises > becomes raises' there is one other defect of my program. It > does not remove the 'import unittest' lines. Or have you already > added that while I was not looking? No, so far we have been focusing on various other aspects, and currently getting nice applevel tracebacks. I guess one always needs to do some manual adjustments so removing "import unittest.py" is not a big issue. For example, with PyPy we use 'from pypy.tool import testit' which needs to go and it is somewhat difficult to delete such lines automatically :-) Getting nice applevel tracebacks requires refactoring towards a more abstract and helpful introspection model (where PyPy can drop it its own more involved one) which is what we have been working on a bit. (Armin is currently out with a friend, though, so he hasn't seen the latest changes :-) cheers, holger From mwh at python.net Sun Dec 19 18:05:00 2004 From: mwh at python.net (Michael Hudson) Date: Sun, 19 Dec 2004 17:05:00 +0000 Subject: [py-dev] Greenlets In-Reply-To: <20041217164707.GL15414@solar.trillke.net> (holger krekel's message of "Fri, 17 Dec 2004 17:47:07 +0100") References: <20041217014755.GE23584@solar.trillke.net> <20041217164707.GL15414@solar.trillke.net> Message-ID: <2mekhmkymb.fsf@starship.python.net> holger krekel writes: > On Fri, Dec 17, 2004 at 12:41 +0100, Florian Schulze wrote: >> On Fri, 17 Dec 2004 02:47:55 +0100, holger krekel wrote: >> > >> >Does installing distutils-based C-coded packages usually work for you? >> >> Yes, if zope3, numarray, pygame etc counts ;) > > OK, Armin worked around an underlying compiler limitation What, that the code wasn't ANSI C to start with? :) Cheers, mwh -- I tried to use woven.guard sunday night and? hazmat: glyph's powers of obfuscation are considerable -- from Twisted.Quotes From hpk at trillke.net Sun Dec 19 18:20:21 2004 From: hpk at trillke.net (holger krekel) Date: Sun, 19 Dec 2004 18:20:21 +0100 Subject: [py-dev] Greenlets In-Reply-To: <2mekhmkymb.fsf@starship.python.net> References: <20041217014755.GE23584@solar.trillke.net> <20041217164707.GL15414@solar.trillke.net> <2mekhmkymb.fsf@starship.python.net> Message-ID: <20041219172021.GF16238@solar.trillke.net> On Sun, Dec 19, 2004 at 17:05 +0000, Michael Hudson wrote: > holger krekel writes: > > > On Fri, Dec 17, 2004 at 12:41 +0100, Florian Schulze wrote: > >> On Fri, 17 Dec 2004 02:47:55 +0100, holger krekel wrote: > >> > > >> >Does installing distutils-based C-coded packages usually work for you? > >> > >> Yes, if zope3, numarray, pygame etc counts ;) > > > > OK, Armin worked around an underlying compiler limitation > > What, that the code wasn't ANSI C to start with? :) Damn, you unscrupulously uncovered the degree of my fading C-knowledge (although i could forward the blame to Armin for that one :) cheers, holger From ksmith at 99hats.com Tue Dec 21 23:47:07 2004 From: ksmith at 99hats.com (Kevin Smith) Date: Tue, 21 Dec 2004 17:47:07 -0500 Subject: [py-dev] How do I access current testing directory? Message-ID: <1103669225.14216@ns5.99hats.com> Great great testing program. What a time saver, very pythonic. My question is how do I access the current working directory (where the test is located) from within the test? Kev From mscott at goldenspud.com Wed Dec 22 03:21:22 2004 From: mscott at goldenspud.com (Matthew Scott) Date: Tue, 21 Dec 2004 20:21:22 -0600 Subject: [py-dev] revision 7936 makes it harder to see failing source compiled from a string Message-ID: <41C8DA22.3040600@goldenspud.com> Love py.test! :) But for now, I find that I am going to revert to revision 7935 and stay at that for a little while 'til the dust settles. Here's why. Take this test which canonically distills the problem: # test_7935_vs_7936.py import imp from py.magic.dyncode import compile import sys def test_it(): src = 'def foo():\n assert 1 == 0\n' name = 'abc-123' module = imp.new_module(name) code = compile(src, name, 'exec') exec code in module.__dict__ sys.modules[name] = module module.foo() If you run it against revision 7936 or higher, you get the following error as the cause of the failure: > /home/gldnspud/p/open/py-dist/py/path/local/local.py, line 516 ------------------------------------------------------------------------------- IOError: << [Errno 2] No such file or directory: '/home/gldnspud/' However, if you use revision 7935, you get this instead: > , line 2 ------------------------------------------------------------------------------- assert 1 == 0 This seems to be happening because the code that is actually raising the exception is inside a module whose code is created using py.magic.dyncode.compile, using a string as the source. I know this isn't common usage of Python, but it is a cornerstone of a project that I'm working on at the moment :) Any ideas on how to work around this? Or could py be changed somehow to accommodate this type of module? Happy with 7935 for now though :) Thanks for the excellent framework! - Matthew From hpk at trillke.net Wed Dec 22 08:37:29 2004 From: hpk at trillke.net (holger krekel) Date: Wed, 22 Dec 2004 08:37:29 +0100 Subject: [py-dev] How do I access current testing directory? In-Reply-To: <1103669225.14216@ns5.99hats.com> References: <1103669225.14216@ns5.99hats.com> Message-ID: <20041222073729.GD19260@solar.trillke.net> Hi Kevin, On Tue, Dec 21, 2004 at 17:47 -0500, Kevin Smith wrote: > Great great testing program. What a time saver, very > pythonic. thanks! > My question is how do I access the current > working directory (where the test is located) from within > the test? mytestfile = py.magic.autopath() gives you the test file itself as a path object. Of course, you could also look at the usual module __file__ string but this would not work for remote path objects. (py.test allows to run tests from svn repositories without downloading etc.pp). You may want to use path objects with datadir = py.magic.autopath().dirpath().join('data') content1 = datadir.join('data1').read() etc.pp. which will of course also work for remote svn urls. I guess i finally need to write some documentation about path objects ... cheers, holger From hpk at trillke.net Wed Dec 22 11:39:05 2004 From: hpk at trillke.net (holger krekel) Date: Wed, 22 Dec 2004 11:39:05 +0100 Subject: [py-dev] revision 7936 makes it harder to see failing source compiled from a string In-Reply-To: <41C8DA22.3040600@goldenspud.com> References: <41C8DA22.3040600@goldenspud.com> Message-ID: <20041222103905.GI19260@solar.trillke.net> On Tue, Dec 21, 2004 at 20:21 -0600, Matthew Scott wrote: > Love py.test! :) great to hear! > But for now, I find that I am going to revert to revision 7935 and stay > at that for a little while 'til the dust settles. i see. > Here's why. Take this test which canonically distills the problem: > > # test_7935_vs_7936.py > import imp > from py.magic.dyncode import compile > import sys > def test_it(): > src = 'def foo():\n assert 1 == 0\n' > name = 'abc-123' > module = imp.new_module(name) > code = compile(src, name, 'exec') > exec code in module.__dict__ > sys.modules[name] = module > module.foo() > > If you run it against revision 7936 or higher, you get the following > error as the cause of the failure: OK, 7935/7936 was indeed the middle of some larger refactoring. I have added your above test so that py.test py/documentation/example/test/failure_demo.py will show the output. Please look at the end of that file to see the way this is done now. (it's basically the same as above but the py.magic.dyncode.* namespace is gone in favour of more higher level introspection/dynamic code helpers in py.code.*). Feedback appreciated although i figure i need to write some documetnation about py.code first unless you are willing to look a bit at the tests especially in py/code/testing/test_source.py ... have fun, holger From pobrien at orbtech.com Wed Dec 22 13:48:12 2004 From: pobrien at orbtech.com (Patrick K. O'Brien) Date: Wed, 22 Dec 2004 06:48:12 -0600 Subject: [py-dev] revision 7936 makes it harder to see failing source compiled from a string In-Reply-To: <41C8DA22.3040600@goldenspud.com> References: <41C8DA22.3040600@goldenspud.com> Message-ID: <41C96D0C.3050704@orbtech.com> Matthew Scott wrote: > Love py.test! :) Ditto. > I know this isn't common usage of Python, but it is a cornerstone of a > project that I'm working on at the moment :) Can you say "Schevo!" :-) > Any ideas on how to work around this? Or could py be changed somehow to > accommodate this type of module? Please, oh please? > Happy with 7935 for now though :) Definitely. > Thanks for the excellent framework! Ditto. -- Patrick K. O'Brien Orbtech http://www.orbtech.com Pypersyst http://www.pypersyst.org Schevo http://www.schevo.org From ianb at colorstudy.com Fri Dec 24 04:35:42 2004 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 24 Dec 2004 03:35:42 +0000 Subject: [py-dev] move tests into their own directory ... In-Reply-To: <20041219161537.GD16238@solar.trillke.net> References: <200412191606.iBJG6l37028773@ratthing-b246.strakt.com> <20041219161537.GD16238@solar.trillke.net> Message-ID: <20041224033542.GA25614@colorstudy.com> On Sun, Dec 19, 2004 at 05:15:38PM +0100, holger krekel wrote: > Eventually i want to get rid of *.pyc files as well in > favour of installing them in a platform specific directory. > The basic idea here is to have completely clean > implementation directories and being able to use > the same py-lib checkout for multiple python versions > effectively (without recompiling all the time). This need > became more apparent, now that we support C-translation > on the fly (e.g. py.magic.greenlet). Have you looked at the PEP about specifying a location for .pyc files? It lays out the mechanism and policy for it -- I can't remember if there was an implementation as well (I think there was). It ended up being dropped because of a lack of interest, but everyone seemed positive about the details it proposed. Ian From kkowalczyk at gmail.com Thu Dec 30 11:46:46 2004 From: kkowalczyk at gmail.com (Krzysztof Kowalczyk) Date: Thu, 30 Dec 2004 02:46:46 -0800 Subject: [py-dev] getstatementrange() broken? Message-ID: <7ce338ad04123002464d24b45c@mail.gmail.com> I'm just starting with py.test so I wrote a trivial program test_foo.py: def test_fail(): # failing test assert 1 == 2 def test_ok(): assert 1 == 1 When I run it with py.test, instead of getting a nice callstack for test_fail() failing assert, I get a long callstack for py.test itself. The problem comes from \py\code\source.py getstatementrange(). Now, I know nothing about py.test but the code there is just weird, e.g.: for end in range(lineno, len(self)): trysource = self[lineno:end+1] if trysource.isparseable(): start = lineno break else: end = lineno I guess I don't know python as well as I thought, but how an "else" to a "for" is even possible? Specifically, the problem comes from the fact, that at some point there is: else: return None and the caller: def getstatement(self, lineno): """ return Source statement which contains the given linenumber (counted from 0). """ > start, end = self.getstatementrange(lineno) doesn't handle that. It must get a sequence. Anyway, things seem to work when I replace the function with just: return lineno, lineno+1 Of course, this is bogus and won't work if lineno is at the end of the file, so someone who knows what's going on in getstatementrange() should take a look. I'm using latest svn sources. Happy New Year to everyone. Krzysztof Kowalczyk : http://blog.kowalczyk.info Here's a full callstack: C:\kjk\src\mine\moriarty_palm\tests>python c:\kjk\src\mine\moriarty_palm\scripts\dist-py\py\bin\py.test ============================= test process starts ============================= testing-mode: inprocess executable : c:\Python24\python.exe (2.4.0-final-0) using py lib: c:\kjk\src\mine\moriarty_palm\scripts\dist-py\py initial testconfig 0: c:\kjk\src\mine\moriarty_palm\scripts\dist-py\py\test\defaultconfig =============================================================================== F. _______________________________________________________________________________ def runitem(self, item): if not self.option.collectonly: self.reporter.startitem(item) try: > item.run(self) [c:\kjk\src\mine\moriarty_palm\scripts\dist-py\py\test\drive.py:92] ------------------------------------------------------------------------------- def run(self, driver): self.setupmanager.setup_path(self.extpy) target, teardown = self.setupmanager.setup_method(self.extpy) try: > self.execute(target, *self.args) [c:\kjk\src\mine\moriarty_palm\scripts\dist-py\py\test\item.py:78] ------------------------------------------------------------------------------- def execute(self, target, *args): """ default implementation for calling a test target is to simply call it. """ > target(*args) [c:\kjk\src\mine\moriarty_palm\scripts\dist-py\py\test\item.py:87] ------------------------------------------------------------------------------- def __init__(self, *args): BuiltinAssertionError.__init__(self, *args) f = sys._getframe(1) try: > source = py.code.Frame(f).statement [c:\kjk\src\mine\moriarty_palm\scripts\dist-py\py\magic\assertion.py:12] ------------------------------------------------------------------------------- def statement(self): > return self.code.fullsource.getstatement(self.lineno) [c:\kjk\src\mine\moriarty_palm\scripts\dist-py\py\code\frame.py:35] ------------------------------------------------------------------------------- def getstatement(self, lineno): """ return Source statement which contains the given linenumber (counted from 0). """ > start, end = self.getstatementrange(lineno) [c:\kjk\src\mine\moriarty_palm\scripts\dist-py\py\code\source.py:62] ------------------------------------------------------------------------------- (reinterpretation failed, you may increase verbosity to see details) TypeError: unpack non-sequence ============= tests finished: 1 passed, 1 failed in 0.02 seconds ============= From hpk at trillke.net Thu Dec 30 13:04:42 2004 From: hpk at trillke.net (holger krekel) Date: Thu, 30 Dec 2004 13:04:42 +0100 Subject: [py-dev] getstatementrange() broken? In-Reply-To: <7ce338ad04123002464d24b45c@mail.gmail.com> References: <7ce338ad04123002464d24b45c@mail.gmail.com> Message-ID: <20041230120442.GD25307@solar.trillke.net> Hi Krzysztof, On Thu, Dec 30, 2004 at 02:46 -0800, Krzysztof Kowalczyk wrote: > I'm just starting with py.test so I wrote a trivial program test_foo.py: > def test_fail(): > # failing test > assert 1 == 2 > > def test_ok(): > assert 1 == 1 ok, this should work and does work on linux & osx. > When I run it with py.test, instead of getting a nice callstack for > test_fail() failing assert, I get a long callstack for py.test itself. yes, this happens when some internal problem arises. > The problem comes from \py\code\source.py getstatementrange(). Now, I > know nothing about py.test but the code there is just weird, e.g.: > for end in range(lineno, len(self)): > trysource = self[lineno:end+1] > if trysource.isparseable(): > start = lineno > break > else: > end = lineno > > I guess I don't know python as well as I thought, but how an "else" to > a "for" is even possible? the else-block is run when the for-loop ran to end. If instead a "break" occurs the else-statement is not run. > Specifically, the problem comes from the fact, that at some point there is: > else: > return None > > and the caller: > def getstatement(self, lineno): > """ return Source statement which contains the > given linenumber (counted from 0). > """ > > start, end = self.getstatementrange(lineno) > doesn't handle that. It must get a sequence. sure. This case should not happen though and if it does an exception is raised. We may modify this to directly raise an exception. > Anyway, things seem to work when I replace the function with just: > return lineno, lineno+1 hardcoding your case works, good :-) > Of course, this is bogus and won't work if lineno is at the end of the > file, so someone who knows what's going on in getstatementrange() > should take a look. > > I'm using latest svn sources. > > Happy New Year to everyone. to you too! > Krzysztof Kowalczyk : http://blog.kowalczyk.info > > Here's a full callstack: hum, i think this was related to not properly reading the source code from the file with "universal newline" support and thus the "getstatementrange" logic didn't work. I just fixed this (rev 8009). Can you retry? if it still doesn't work can you rerun your failing test with "--nomagic --fulltrace --showlocals" and send the output? thanks & cheers, holger From kkowalczyk at gmail.com Fri Dec 31 01:45:28 2004 From: kkowalczyk at gmail.com (Krzysztof Kowalczyk) Date: Thu, 30 Dec 2004 16:45:28 -0800 Subject: [py-dev] getstatementrange() broken? In-Reply-To: <20041230120442.GD25307@solar.trillke.net> References: <7ce338ad04123002464d24b45c@mail.gmail.com> <20041230120442.GD25307@solar.trillke.net> Message-ID: <7ce338ad041230164526c3948@mail.gmail.com> > hum, i think this was related to not properly reading > the source code from the file with "universal newline" > support and thus the "getstatementrange" logic didn't > work. I just fixed this (rev 8009). Can you retry? Yes, the change to frame.py fixed it for me. Thanks for speedy fix. Krzysztof Kowalczyk : http://blog.kowalczyk.info