From jopython at gmail.com Mon May 5 17:01:44 2008 From: jopython at gmail.com (Joe Python) Date: Mon, 5 May 2008 11:01:44 -0400 Subject: [py-dev] Building greenlet 0.9.1 in Solaris 10 Message-ID: Building greenlet 0.9.1 in Solaris 10 While trying to build the c extension within Solaris 10, i get the following error message: bash-3.00# /usr/local/bin/python setup.py build running build running build_ext building 'greenlet' extension creating build creating build/temp.solaris-2.10-sun4u-2.5 gcc -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python2.5 -c greenlet.c -o build/temp.solaris-2.10-sun4u-2.5/greenlet.o In file included from slp_platformselect.h:16, from greenlet.c:251: switch_sparc_sun_gcc.h: In function `slp_switch': switch_sparc_sun_gcc.h:56: error: `_cst' undeclared (first use in this function) switch_sparc_sun_gcc.h:56: error: (Each undeclared identifier is reported only once switch_sparc_sun_gcc.h:56: error: for each function it appears in.) switch_sparc_sun_gcc.h:52: error: PIC register `%l7' clobbered in `asm' error: command 'gcc' failed with exit status 1 bash-3.00# gcc -v Reading specs from /usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/specs Configured with: /gates/sfw10/builds/sfw10-gate/usr/src/cmd/gcc/gcc-3.4.3/configure --prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as --with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++ --enable-shared Thread model: posix gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath) Python build is : Python 2.5.1 (r251:54863, May 16 2007, 19:58:05) [GCC 3.4.6] on sunos5 bash-3.00# cat /etc/release Solaris 10 3/05 s10_74L2a SPARC -------------- next part -------------- An HTML attachment was scrubbed... URL: From lijssies at gp-net.nl Tue May 6 08:34:44 2008 From: lijssies at gp-net.nl (Gerard Petersen) Date: Tue, 06 May 2008 08:34:44 +0200 Subject: [py-dev] Request for enhancing Exception handling in execnet Message-ID: <200805060834.44919.lijssies@gp-net.nl> Hi All, I write this email as a follow up to a chat on #pylib with cfbolz. Bare with me in the code example and explanation because I'm only into python for six months or so. The reason for this request is that I'm building an application with wx.Python that does remote file handling based on execnet. I was told that it's good practice to have exceptions raised in your code as low as possible, and handle them as high in your code as possible. A brief setup of my app (just to illustrate) gui/mainapp (the gui) dwlib/remote (the remote module) In the app (the gui) I call a function from dwlib that gets a remote dirlisting. If this fails the gui pops up a dialog with advice on how to continu. As the docs of execnet state. Errors on the remote side are raised as OSErrors and contain a textual version of the remote exception. Two that I forced (simulated): OSError: [Errno 13] Permission denied: '/some/where' OSError: [Errno 2] No such file or directory: '/some/whe' This is correct behaviour but I would like to have them as actual exceptions back up to the interface. In this case I have to parse the string in the dwlib module with something like "if value.startswith(OSError')" and raise another exception my self. So the request is: Is it possible to actually have the Exception from the remote side passed on as an exception object to the local side (or recreate one?!?). Attached is a script that simulates the problem. There are SSH keys in place, so (local)host access is functioning. As said I'm not a python guru (yet). so please don't shoot my code ... :-) I hope all is clear. Questions on the mather (and tips) are very welcome. Kind regards, Gerard. -- $Grtz =~ Gerard; ~ :wq! -------------- next part -------------- A non-text attachment was scrubbed... Name: fileread.py Type: application/x-python Size: 939 bytes Desc: not available URL: From johnny at johnnydebris.net Tue May 6 09:38:20 2008 From: johnny at johnnydebris.net (Guido Wesdorp) Date: Tue, 06 May 2008 09:38:20 +0200 Subject: [py-dev] Request for enhancing Exception handling in execnet In-Reply-To: <200805060834.44919.lijssies@gp-net.nl> References: <200805060834.44919.lijssies@gp-net.nl> Message-ID: <48200AEC.40505@johnnydebris.net> Gerard Petersen wrote: > Errors on the remote side are raised > as OSErrors and contain a textual version of the remote exception. They are actually raised as RemoteError (py.__.execnet.channel.RemoteError) instances, but I understand the frustration. The RemoteError instance doesn't hold a reference to the original error, which can be annoying. I think one of the problems could be that the remote exception may not be valid on the local machine: for instance if you import a library 'foo' that raises a (self-invented) FooException on the remote side, that library may not be available locally so it may be that FooException can not be instantiated/raised there. I'm not sure how to deal with this problem. I do know execnet has the possibility to pickle (?) and send over exceptions, though... I think what you could do in your code is catch the specific exceptions you're interested in on the remote side yourself, and send those over the channel if something goes wrong, rather than the data you would have sent if it would have succeeded. Of course this means that both the client and the server (never sure which is which, though ;) will become more complex, but I don't think it's too bad: g = py.execnet.PopenGateway() g.execute("""\ import os try: l = os.listdir('/some/dir/that/may/have/problems') except OSError, e: # could even be 'Exception' instead of 'OSError' channel.send(e) else: channel.send(l) """) ret = channel.receive() if isinstance(ret, Exception): raise ret # raise the remote exception again assert isinstance(ret, list) # here you know 'ret' is a list I understand this isn't optimal, and doesn't solve the problem entirely (just for the exceptions you deal with explicitly), but at least you can now get back to work again... ;) Cheers, Guido From cfbolz at gmx.de Tue May 6 09:45:03 2008 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 06 May 2008 09:45:03 +0200 Subject: [py-dev] Request for enhancing Exception handling in execnet In-Reply-To: <48200AEC.40505@johnnydebris.net> References: <200805060834.44919.lijssies@gp-net.nl> <48200AEC.40505@johnnydebris.net> Message-ID: <48200C7F.3030202@gmx.de> Guido Wesdorp wrote: > Gerard Petersen wrote: >> Errors on the remote side are raised >> as OSErrors and contain a textual version of the remote exception. > They are actually raised as RemoteError > (py.__.execnet.channel.RemoteError) instances, but I understand the > frustration. The RemoteError instance doesn't hold a reference to the > original error, which can be annoying. > > I think one of the problems could be that the remote exception may not > be valid on the local machine: for instance if you import a library > 'foo' that raises a (self-invented) FooException on the remote side, > that library may not be available locally so it may be that FooException > can not be instantiated/raised there. I'm not sure how to deal with this > problem. I do know execnet has the possibility to pickle (?) and send > over exceptions, though... Execnet is using only marshal, afaik. However, it should be possible to be at least a bit more informative and give RemoteError a name attribute or so that gives the name of the remote exception class. Just giving some message with traceback and everything makes it annoying to do anything with the exception object programmatically. Cheers, Carl Friedrich From johnny at johnnydebris.net Tue May 6 12:54:02 2008 From: johnny at johnnydebris.net (Guido Wesdorp) Date: Tue, 06 May 2008 12:54:02 +0200 Subject: [py-dev] Request for enhancing Exception handling in execnet In-Reply-To: <48200C7F.3030202@gmx.de> References: <200805060834.44919.lijssies@gp-net.nl> <48200AEC.40505@johnnydebris.net> <48200C7F.3030202@gmx.de> Message-ID: <482038CA.4090504@johnnydebris.net> Carl Friedrich Bolz wrote: > Execnet is using only marshal, afaik. However, it should be possible to > be at least a bit more informative and give RemoteError a name attribute > or so that gives the name of the remote exception class. Just giving > some message with traceback and everything makes it annoying to do > anything with the exception object programmatically. > > Yes, I agree... Although I still think it's not easy to find a good way to send all the information (exception instance + traceback would be nice obviously) back over the channel, it would certainly be useful to do the minimal (set seperate instance name + message on the RemoteError instance for instance) compared to having a single explanation string... Cheers, Guido From faassen at startifact.com Tue May 6 16:34:45 2008 From: faassen at startifact.com (Martijn Faassen) Date: Tue, 06 May 2008 16:34:45 +0200 Subject: [py-dev] MAX_PATH on windows Message-ID: Hi there, Apparently the Windows API has a limitation in how long paths can get, set to 256 characters!(!!) This can break applications on Windows, including Py. Here are some relevant resources: http://mail.python.org/pipermail/python-bugs-list/2006-June/033754.html http://mail.python.org/pipermail/python-bugs-list/2007-March/037810.html http://msdn.microsoft.com/en-us/library/aa365247.aspx There is some discussion that this could be fixed in Python 2.5. It's also however not 100% clear to me, looking at the discussions, whether it really is. I'm on Python 2.4 myself... I think at the very least it'd be good to have a test case in Py that tries to trigger this error. Apparently a workaround is to prefix windows paths by "\\?\". Paths can then be a lot longer. I'm not sure how this interacts with the Python use of these APIs, however. If the workaround would really work, py could generate its windows paths with that prefix. Regards, Martijn From lijssies at gp-net.nl Wed May 7 08:30:29 2008 From: lijssies at gp-net.nl (Gerard Petersen) Date: Wed, 07 May 2008 08:30:29 +0200 Subject: [py-dev] Request for enhancing Exception handling in execnet In-Reply-To: <482038CA.4090504@johnnydebris.net> References: <200805060834.44919.lijssies@gp-net.nl> <48200C7F.3030202@gmx.de> <482038CA.4090504@johnnydebris.net> Message-ID: <200805070830.29349.lijssies@gp-net.nl> Guido, Thanx for the code snippet. Initially I wanted to keep the code that goes remote as clean as possible (somebody told me ... ;-) but the code you came up with does seem good. The actual issue is somewhat more elaborate. I also have remote functions that read files, and a predicatble situation is that a file might not be there remotely. Since this is a possibiltity and not an exception as such, (next to a "permission denied" which is). There is also the possibility to return an error value along all the time. You then would still need some error handling remotely but the last line in your function would be something like "return contents, error". "Error" containing a level from zero upwards. I'll see if I can combine a setup like that with the "isinstance" function. More on the way, Thanx guys!! Regards, Gerard. On Tuesday 06 May 2008 12:54:02 Guido Wesdorp wrote: > Carl Friedrich Bolz wrote: > > Execnet is using only marshal, afaik. However, it should be possible to > > be at least a bit more informative and give RemoteError a name attribute > > or so that gives the name of the remote exception class. Just giving > > some message with traceback and everything makes it annoying to do > > anything with the exception object programmatically. > > Yes, I agree... Although I still think it's not easy to find a good way > to send all the information (exception instance + traceback would be > nice obviously) back over the channel, it would certainly be useful to > do the minimal (set seperate instance name + message on the RemoteError > instance for instance) compared to having a single explanation string... > > Cheers, > > Guido -- $Grtz =~ Gerard; ~ :wq! From walter at livinglogic.de Wed May 7 12:04:29 2008 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed, 07 May 2008 12:04:29 +0200 Subject: [py-dev] Request for enhancing Exception handling in execnet In-Reply-To: <200805070830.29349.lijssies@gp-net.nl> References: <200805060834.44919.lijssies@gp-net.nl> <48200C7F.3030202@gmx.de> <482038CA.4090504@johnnydebris.net> <200805070830.29349.lijssies@gp-net.nl> Message-ID: <48217EAD.60905@livinglogic.de> Gerard Petersen wrote: > Guido, > > Thanx for the code snippet. Initially I wanted to keep the code that goes > remote as clean as possible (somebody told me ... ;-) but > the code you came up with does seem good. > > The actual issue is somewhat more elaborate. I also have remote functions that > read files, and a predicatble situation is that a file might not be there > remotely. Since this is a possibiltity and not an exception as such, (next to > a "permission denied" which is). There is also the possibility to return an > error value along all the time. You then would still need some error handling > remotely but the last line in your function would be something like "return > contents, error". > "Error" containing a level from zero upwards. I'll see if I can combine a > setup like that with the "isinstance" function. > > More on the way, Thanx guys!! You might take a look at the ll.url module which is part of XIST (available from: http://www.livinglogic.de/Python/Download.html ll.url implements an ssh URL scheme which supports all file operations remotely: >>> from ll import url >>> u = url.URL("ssh://root at www.example.org/~/") >>> u.listdir(context=url.Context()) [URL('.bash_history'), URL('.profile'), URL('.bashrc'), ... >>> u = url.URL("ssh://root at www.example.org/~/.bash_history") >>> f = u.open("rb", context=url.Context()) >>> f.readline() '/etc/init.d/tomcat stop\n' >>> f.readline() 'ps waux | grep tomcat\n' It mirrors exceptions locally: >>> u = url.URL("ssh://root at www.example.org/~/does-not-exist") >>> f = u.open("rb", context=url.Context()) [...] IOError: [Errno 2] No such file or directory: '/root/does-not-exist' (of course this only works for builtin exceptions) And of course ll.url uses py.execnet for communicating with the remote host. Hope that helps! Servus, Walter