From kbk at shore.net Tue Feb 1 04:44:08 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Tue Feb 1 04:44:18 2005 Subject: [Idle-dev] script changed by another process In-Reply-To: <20050129182855.39536.qmail@web52805.mail.yahoo.com> (paul breen's message of "Sat, 29 Jan 2005 10:28:55 -0800 (PST)") References: <20050129182855.39536.qmail@web52805.mail.yahoo.com> Message-ID: <87r7k10ws7.fsf@hydra.bayview.thirdcreek.com> paul breen writes: > I like to use my favorite editor to make most changes to a script then > switch to IDLE to run the program. The trouble is that idle does not > detect that the file has been changed like most editors do. Usually a > code editor will ask to reload the file. Idle will not reflect the > change even if the file is opened again within the same window. The > only way to get around the problem is to close the window and re-open > the file in a new window. Are there plans to change to change this > behavior? This sounds familiar, though I don't see a Bug in either Python or IDLEfork. It's not high on my list. Fixing it is not too big an issue, it would require adding a timestamp attribute to the EditorWindow class and comparing it to the file on disk before running or saving. Maybe you could provide a patch. Otherwise enter a Bug in the Python tracker and maybe we'll get to it someday. Why not use IDLE to edit the code? It's very convenient, with one keystroke save, syntax check, and run in shell. -- KBK From tpc at csua.berkeley.edu Tue Feb 15 03:38:08 2005 From: tpc at csua.berkeley.edu (tpc@csua.berkeley.edu) Date: Tue Feb 15 03:36:42 2005 Subject: [Idle-dev] IDLE freezing/hanging Message-ID: <20050214183109.M29323-100000@localhost.name> hello, have any of you experienced this ? This has been going on intermittently for quite some time, I have no idea what causes it. What can I do to help debug the problem or find out the cause ? Enclosed below is a more or less accurate log of IDLE freezes, and as you can see I went for a few months where it didn't crash. I am wondering if this all has to do with how IDLE interacts with Debian system. I am on Debian unstable and I do once a week apt-get updates and apt-get dist-upgrades. 15OCT2004 2:06pm while testing code to see if convertListToString is necessary Debian Sarge Python IDLE just froze on me again pressing keys does nothing, unresponsive, total loss of data second time this has happened to me only thing I can do is kill -9 the process and it pisses me off, this is the third time I can remember where Python IDLE just crashes on me, just freezes up and is non-responsive and I have to kill -9 the process and reboot and reload everything I've done up to this point this occurred while I was typing up experiment re looping through two lists where len(eachList) was different, what would happen in such a case ? 01NOV2004 10:29am IDLE froze, apt-get removed, rebooted system, then apt-get installed 01NOV2004 11:43am IDLE just crashed again 12NOV2004 1:57pm ok, so after another event where IDLE froze, I must say I am very dissatisfied with this Debian install, because the copy paste between xemacs buffers leaves much to be desired and the IDLE freezes for no apparent reason. I will have to do another migration. 14FEB2005 3:25pm just tooling around on IDLE and it froze, now I have to kill -9 the process and restart it 14FEB2005 6:24pm IDLE froze again, just when I was getting to some test runs AARRGH! This hang occurred after I upgraded to IDLE 2.3.5-1 after the last hang. From john.zelle at wartburg.edu Tue Feb 15 03:55:36 2005 From: john.zelle at wartburg.edu (John Zelle) Date: Tue Feb 15 03:58:44 2005 Subject: [Idle-dev] IDLE freezing/hanging In-Reply-To: <20050214183109.M29323-100000@localhost.name> References: <20050214183109.M29323-100000@localhost.name> Message-ID: <421164A8.8030904@wartburg.edu> Hello, This is just a quick "me too." I am running Debian testing (Sarge), and IDLE is terribly unstable. It hangs frequently while in the middle of typing. I thought this was probably a library problem due to some mixing of distributions that I've done, but I now have a "pure" Sarge install and am seeing exactly this same behavior. I have basically told my students to use Emacs or Eclipse instead. A shame, because I really like IDLE, when it works. Is this just a Debian issue, or is it a general problem on Linux (Tk library, perhaps?). --John tpc@csua.berkeley.edu wrote: > hello, have any of you experienced this ? This has been going on > intermittently for quite some time, I have no idea what causes it. What > can I do to help debug the problem or find out the cause ? Enclosed below > is a more or less accurate log of IDLE freezes, and as you can see I went > for a few months where it didn't crash. I am wondering if this all has to > do with how IDLE interacts with Debian system. I am on Debian unstable > and I do once a week apt-get updates and apt-get dist-upgrades. > > 15OCT2004 2:06pm > while testing code to see if convertListToString is necessary > Debian Sarge Python IDLE just froze on me again > pressing keys does nothing, unresponsive, total loss of data > second time this has happened to me > only thing I can do is kill -9 the process > > and it pisses me off, this is the third time I can remember where Python > IDLE just crashes on me, just freezes up and is non-responsive and I have > to kill -9 the process and reboot and reload everything I've done up to > this point > this occurred while I was typing up experiment re looping through two > lists where len(eachList) was different, what would happen in such a case > ? > 01NOV2004 10:29am IDLE froze, apt-get removed, rebooted system, then > apt-get installed > > 01NOV2004 11:43am IDLE just crashed again > > 12NOV2004 1:57pm > ok, so after another event where IDLE froze, I must say I am very > dissatisfied with this Debian install, because the copy paste between > xemacs buffers leaves much to be desired and the IDLE freezes for no > apparent reason. I will have to do another migration. > > 14FEB2005 3:25pm > just tooling around on IDLE and it froze, now I have to kill -9 the > process and restart it > > 14FEB2005 6:24pm > IDLE froze again, just when I was getting to some test runs AARRGH! This > hang occurred after I upgraded to IDLE 2.3.5-1 after the last hang. > > > _______________________________________________ > IDLE-dev mailing list > IDLE-dev@python.org > http://mail.python.org/mailman/listinfo/idle-dev > > From dooms at info.ucl.ac.be Tue Feb 15 10:42:12 2005 From: dooms at info.ucl.ac.be (=?ISO-8859-1?Q?Gr=E9goire_Dooms?=) Date: Tue Feb 15 10:45:13 2005 Subject: [Idle-dev] IDLE freezing/hanging In-Reply-To: <421164A8.8030904@wartburg.edu> References: <20050214183109.M29323-100000@localhost.name> <421164A8.8030904@wartburg.edu> Message-ID: <4211C3F4.4070506@info.ucl.ac.be> Me too. I use Debian unstable and Idle patched with Raphael Noam's enhancements (they are great I would love to see this merged). I tried to debug the problem but am not aware of a way to attach a debugger to a python process (I would like to do it at a higher level than interpreter level with gdb). All I could do up to now is strace both processes and see one of them repeating a sequence of futex and select calls (this is normal I think) while the other is blocked in a futex call. I tried to get that process out of the futex call by sending a signal but all signals I tried up to now made it exit (HUP and CHLD, if I remember correctly). The problem is not easily reproductible as it seems to occur at random. But I feel it could be related to rate of typing as I nearly always get it when typing on my laptop keyboard. I did not submit a bug as this is patched code and I never had a student annoyed by this bug. I'm willing to help trace and fix the bug if you can direct me in the right direction. -- Gr?goire Dooms From davidschein at alumni.tufts.edu Wed Feb 16 21:51:47 2005 From: davidschein at alumni.tufts.edu (David S.) Date: Wed Feb 16 22:03:11 2005 Subject: [Idle-dev] subprocess problem on Windows in IDLE Message-ID: please see: http://thread.gmane.org/gmane.comp.python.general/388373 From kbk at shore.net Thu Feb 17 21:37:34 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu Feb 17 21:38:14 2005 Subject: [Idle-dev] subprocess problem on Windows in IDLE In-Reply-To: (David S.'s message of "Wed, 16 Feb 2005 20:51:47 +0000 (UTC)") References: Message-ID: <87r7jeudm9.fsf@hydra.bayview.thirdcreek.com> "David S." writes: > please see: > http://thread.gmane.org/gmane.comp.python.general/388373 Python Bug 1126208 ===================== From: David S. alumni.tufts.edu> Subject: subprocess problem on Windows in IDLE and PythonWin Newsgroups: gmane.comp.python.general Date: Wed, 16 Feb 2005 02:05:24 +0000 Python 2.4 on Windows XP In the python command-line the following works fine: >>> from subprocess import * >>> p = Popen('dir', stdout=PIPE) >From within IDLE or PythonWin I get the following exception: Traceback (most recent call last): File "", line 1, in -toplevel- p = Popen('dir', stdout=PIPE) File "c:\python24\lib\subprocess.py", line 545, in __init__ (p2cread, p2cwrite, File "c:\python24\lib\subprocess.py", line 605, in _get_handles p2cread = self._make_inheritable(p2cread) File "c:\python24\lib\subprocess.py", line 646, in _make_inheritable DUPLICATE_SAME_ACCESS) TypeError: an integer is required Note it works fine on Linux also. I tested it with >>> p = Popen('ls', stdout=PIPE) ... and had no trouble. =========== I (KBK) can duplicate this on W2K. If I run IDLE with the -n switch (no subprocess) the error doesn't occur. Unfortunately, I can't debug it because I don't have the necessary tools on Windows. It appears that the problem is in _subprocess.c:sp_DuplicateHandle(), likely that PyArg_ParseTuple() is OK but the failure occurs in the call to DuplicateHandle(). All the args to sp_DuplicateHandle() seem to be the right type. DUPLICATE_SAME_ACCESS is an integer, value 2 To find out what's going on, it would seem necessary to attach a windows debugger to IDLE's subprocess (not the IDLE GUI). Let me know if I can help. =========================== Suggest you monitor that Bug. -- KBK From anthra.norell at tiscalinet.ch Tue Feb 22 11:11:25 2005 From: anthra.norell at tiscalinet.ch (Anthra Norell) Date: Wed Feb 23 14:55:04 2005 Subject: [Idle-dev] IDLE output too slow Message-ID: <01d001c518c6$de8e8300$0201a8c0@mcuf7> I posted the follwing message on python.sig and there was no response. Hopefully you can offer some advice. Thanks very much. - Frederic Hi, I upgraded from 2.2.2 to 2.4 and all is well except the output to the IDLE window is now twenty times slower than it was before, making the window utterly unusable for verbose output. The statement -- for i in range (100): print i -- now takes about forty-five seconds to complete! Used to be two. Python 2.4 on Windows ME. Hints will be gratefully received by Frederic -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/idle-dev/attachments/20050222/a37c2e7b/attachment.htm From anthra.norell at tiscalinet.ch Tue Feb 22 14:44:24 2005 From: anthra.norell at tiscalinet.ch (Anthra Norell) Date: Wed Feb 23 14:55:04 2005 Subject: [Idle-dev] IDLE output speed problem Message-ID: <00bc01c518e4$9f584720$0201a8c0@mcuf7> I posted the follwing message on python.sig and there was no response. I hope to have more luck here. Thanks very much. - Frederic Hi, I upgraded from 2.2.2 to 2.4 and all is well except the output to the IDLE window is now twenty times slower than it was before, making the window utterly unusable for verbose output. The statement -- for i in range (100): print i -- now takes about forty-five seconds to complete! Used to be two. Python 2.4 on Windows ME. Hints will be gratefully received by Frederic -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/idle-dev/attachments/20050222/48a5bbbe/attachment.html From kbk at shore.net Wed Feb 23 19:04:01 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed Feb 23 19:04:19 2005 Subject: [Idle-dev] IDLE output speed problem In-Reply-To: <00bc01c518e4$9f584720$0201a8c0@mcuf7> (Anthra Norell's message of "Tue, 22 Feb 2005 14:44:24 +0100") References: <00bc01c518e4$9f584720$0201a8c0@mcuf7> Message-ID: <871xb7nofi.fsf@hydra.bayview.thirdcreek.com> "Anthra Norell" writes: > I posted the follwing message on python.sig and there was no > response. I hope to have more luck here. Thanks very much. - > Frederic > > Hi, I upgraded from 2.2.2 to 2.4 and all is well except the output > to the IDLE window is now twenty times slower than it was before, > making the window utterly unusable for verbose output. The statement > -- for i in range (100): print i -- now takes about forty-five > seconds to complete! Used to be two. Python 2.4 on Windows ME. IDLE changed quite a bit at 2.3, user code is now executed in a subprocess connected to the GUI via sockets. But I have no idea why you are having this problem. On Windows 2000 (using 2.4 at about P2/300MHz) I get: for i in range(100): print i ==> less than two seconds for i in range(1000): print i ==> 19 seconds for i in range(1000): print i, ==> 60 seconds The slowdown for the last example is due to Tk issues with printing long lines, i.e. that was one long line, while the other two are individual lines. Try running IDLE with the -n switch, either from the command line or by changing the target in the Run / Python / IDLE menu. Then IDLE runs the same way it did in 2.2, without the subprocess. If that fixes the problem, there may be an issue with Windows ME sockets. Try upgrading (or even downgrading to W98 :-) Overall, as I recollect, the new IDLE is about 30% slower than the old on Linux, OpenBSD, W2K, and WXP, due to the socket communication. But taking advantage of Moore's law, the overall effect since 2.2 has been a speedup ;-) At least we didn't use all the cycles, like Gates does. (I had a jerky Flight Simulator on an Apple II+, now I've got a jerky Flight Simulator on WXP 2.4 GHz, except (default install) it has nice pink fractal clouds and realistic scenery instead of vector graphics.) I wonder if anyone else on the list is using Windows ME with IDLE? -- KBK From kbk at shore.net Wed Feb 23 20:04:25 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed Feb 23 20:04:36 2005 Subject: [Idle-dev] IDLE freezing/hanging In-Reply-To: <4211C3F4.4070506@info.ucl.ac.be> =?iso-8859-1?q?=28Gr=E9goire?= Dooms's message of "Tue, 15 Feb 2005 10:42:12 +0100") References: <20050214183109.M29323-100000@localhost.name> <421164A8.8030904@wartburg.edu> <4211C3F4.4070506@info.ucl.ac.be> Message-ID: <87wtszm72e.fsf@hydra.bayview.thirdcreek.com> Gr?goire Dooms writes: > Me too. > I use Debian unstable and Idle patched with Raphael Noam's > enhancements (they are great I would love to see this merged). This will happen fairly soon. He has an enthusiatic following. > I tried to debug the problem but am not aware of a way to attach a > debugger to a python process (I would like to do it at a higher level > than interpreter level with gdb). There is a graphics version of gdb: ddd http://www.gnu.org/software/ddd/ddd.html > All I could do up to now is strace both processes and see one of them > repeating a sequence of futex and select calls (this is normal I think) > while the other is blocked in a futex call. OK, it may be that the subprocess is somehow blocked looking for input from the GUI. > I tried to get that process out of the futex call by sending a signal > but all signals I tried up to now made it exit (HUP and CHLD, if I > remember correctly). The problem is not easily reproductible as it > seems to occur at random. But I feel it could be related to rate of > typing as I nearly always get it when typing on my laptop keyboard. The blizzard of futex calls is due to polling the sockets. That is done by both the GUI and the subprocess (execution server). The tricky thing debugging IDLE is to do it amid all that activity. If you can get to a point where you have something at least partially reproducible, try running IDLE with the -n switch : (cd ..../Lib/idlelib && ../../python ./PyShell.py -n) or idle -n, depending on how your system is set up. See if the problem goes away. > I did not submit a bug as this is patched code and I never had a > student annoyed by this bug. > I'm willing to help trace and fix the bug if you can direct me in the > right direction. One thing you can try is to turn on the tracing built into IDLE's rpc.py: File / Open Module : rpc In class RPCHandler and class RPCClient there are 'debugging' variables. Set them to True and restart IDLE. You'll now see extensive output in the command window. Note that this output is an interleave of everything going on, and, as is typical of output from several threads, may be slightly out of sequence. But there are sequence numbers associated with each rpc call to help you decode all that. If that isn't enough, there are some print>>sys.__stderr__ statements in rpc.py which are commented out. You can turn them on to get a veritable blizzard of traces :-) When IDLE freezes, I assume you mean that the GUI is unresponsive, and that you can't restart the shell from the Shell menu. If so, try this: kill the subprocess with 'kill -KILL xxxxx'. The subprocess is the one with '8833' in its command line (ps ax). IDLE should then spawn a fresh subprocess and you should be able to continue without losing any of your work. If it doesn't, that points to a problem with Tk. Does it happen while you're typing quickly, or will it freeze while you're having lunch? I must say: I've been using IDLE for years, on W98, W2K, WXP, RH6.2, Arch Linux, and OpenBSD. I've never had IDLE freeze on me unless I made a coding error while working on IDLE itself. Then I see a traceback in the command window. I have had X crash. And even a couple of emacs hangs. These things seem to happen once in a blue moon when I'm using Gnome on Arch (which is pretty bleeding edge). What's your desktop/window manager? (I mostly use Ion, it works extremely well with IDLE because Ion uses non-overlapping panes and each pane can contain multiple tabbed IDLE windows which can be dragged/dropped into different panes. After you use Ion for awhile you realize that overlapping windows are just an awful design choice made at Xerox PARC lo those many years ago. It's also super lightweight and stable.) IDLE is written in pure Python. So if the 'freeze' isn't due to some hitherto undetected GUI/subprocess interaction deadlock (which can be resolved w/o losing work by killing the subprocess), then there is an issue in the C code implementing Python or Tk. Hardware seems to be ruled out because all of you are having problems on Debian Sid. And the way to debug that is to attach gdb to it. It won't be pretty. The subprocess has two threads: one executing user code, and one minding the sockets. The GUI has one thread which executes the Tk event loop, with polling implemented using after() calls. Additional threads are created as necessary for callbacks from the subprocess. -- KBK From john.zelle at wartburg.edu Thu Feb 24 21:43:33 2005 From: john.zelle at wartburg.edu (John Zelle) Date: Thu Feb 24 21:45:03 2005 Subject: [Idle-dev] IDLE freezing/hanging In-Reply-To: <87wtszm72e.fsf@hydra.bayview.thirdcreek.com> References: <20050214183109.M29323-100000@localhost.name> <421164A8.8030904@wartburg.edu> <4211C3F4.4070506@info.ucl.ac.be> <87wtszm72e.fsf@hydra.bayview.thirdcreek.com> Message-ID: <421E3C75.7050903@wartburg.edu> I haven't had a chance yet to try out Kurt's debugging suggestions, but I have attached strace. This hang is so bad on my platform (Debian testing, 2.6.10 stock [non-debian] kernel, KDE 3.3) as to render IDLE unuseable. All I have to do is fire up IDLE and start doing any kind of editing on a new file or an existing one, and in very short order, the process locks up. Somtimes I am typing, sometimes just when I leave it idleing. I have not noticed that typing rate is a factor. The strace sequence always ends up the same, I'm attaching one example run. It always dies with a FUTEX_WAKE, FUTEX_WAIT, FUTEX_WAKE sequence, with strace not printing the complete final call. I have no idea what that means. Perhaps someone who knows more can make sense of it. When I get a chance, I'll run some more experiments along the lines suggested below. The really strange part of this is that I had this problem before, and _somehow_ managed to get IDLE useable again. I thought it was a library upgrade that fixed it. But I now have a clean install with all libs up-t0-date, and it's worse than ever. I am going to try building from the Python source to see if this could be a problem with the Debian binary distribution. --John Zelle Kurt B. Kaiser wrote: > Gr?goire Dooms writes: > > >>Me too. >>I use Debian unstable and Idle patched with Raphael Noam's >>enhancements (they are great I would love to see this merged). > > > This will happen fairly soon. He has an enthusiatic following. > > >>I tried to debug the problem but am not aware of a way to attach a >>debugger to a python process (I would like to do it at a higher level >>than interpreter level with gdb). > > > There is a graphics version of gdb: ddd > > http://www.gnu.org/software/ddd/ddd.html > > >>All I could do up to now is strace both processes and see one of them >>repeating a sequence of futex and select calls (this is normal I think) >> while the other is blocked in a futex call. > > > OK, it may be that the subprocess is somehow blocked looking for input > from the GUI. > > >>I tried to get that process out of the futex call by sending a signal >>but all signals I tried up to now made it exit (HUP and CHLD, if I >>remember correctly). The problem is not easily reproductible as it >>seems to occur at random. But I feel it could be related to rate of >>typing as I nearly always get it when typing on my laptop keyboard. > > > The blizzard of futex calls is due to polling the sockets. That is > done by both the GUI and the subprocess (execution server). The tricky > thing debugging IDLE is to do it amid all that activity. > > If you can get to a point where you have something at least partially > reproducible, try running IDLE with the -n switch : > > (cd ..../Lib/idlelib && ../../python ./PyShell.py -n) or idle -n, > depending on how your system is set up. See if the problem goes away. > > >>I did not submit a bug as this is patched code and I never had a >>student annoyed by this bug. >>I'm willing to help trace and fix the bug if you can direct me in the >>right direction. > > > One thing you can try is to turn on the tracing built into IDLE's > rpc.py: > > File / Open Module : rpc > > In class RPCHandler and class RPCClient there are 'debugging' > variables. Set them to True and restart IDLE. You'll now see > extensive output in the command window. Note that this output is an > interleave of everything going on, and, as is typical of output from > several threads, may be slightly out of sequence. But there are > sequence numbers associated with each rpc call to help you decode all > that. If that isn't enough, there are some print>>sys.__stderr__ > statements in rpc.py which are commented out. You can turn them on to > get a veritable blizzard of traces :-) > > When IDLE freezes, I assume you mean that the GUI is unresponsive, and > that you can't restart the shell from the Shell menu. If so, try > this: kill the subprocess with > > 'kill -KILL xxxxx'. The subprocess is the one with '8833' in its > command line (ps ax). > > IDLE should then spawn a fresh subprocess and you should be able to > continue without losing any of your work. > > If it doesn't, that points to a problem with Tk. Does it happen while > you're typing quickly, or will it freeze while you're having lunch? > > I must say: I've been using IDLE for years, on W98, W2K, WXP, RH6.2, > Arch Linux, and OpenBSD. I've never had IDLE freeze on me unless I > made a coding error while working on IDLE itself. Then I see a > traceback in the command window. > > I have had X crash. And even a couple of emacs hangs. These things > seem to happen once in a blue moon when I'm using Gnome on Arch (which > is pretty bleeding edge). What's your desktop/window manager? > > (I mostly use Ion, it works extremely well with IDLE because Ion uses > non-overlapping panes and each pane can contain multiple tabbed IDLE > windows which can be dragged/dropped into different panes. After you > use Ion for awhile you realize that overlapping windows are just an > awful design choice made at Xerox PARC lo those many years ago. It's > also super lightweight and stable.) > > IDLE is written in pure Python. So if the 'freeze' isn't due to some > hitherto undetected GUI/subprocess interaction deadlock (which can be > resolved w/o losing work by killing the subprocess), then there is an > issue in the C code implementing Python or Tk. Hardware seems to be > ruled out because all of you are having problems on Debian Sid. > > And the way to debug that is to attach gdb to it. It won't be pretty. > The subprocess has two threads: one executing user code, and one > minding the sockets. The GUI has one thread which executes the Tk > event loop, with polling implemented using after() calls. Additional > threads are created as necessary for callbacks from the subprocess. > -------------- next part -------------- ioctl(3, FIONREAD, [32]) = 0 read(3, "\36\360\6\v\26\36:\0^\0\0\3\22\0\200\1\1\0\0\0\37\0\0\0"..., 32) = 32 gettimeofday({1108935078, 484691}, {360, 0}) = 0 select(6, [5], [], [], {0, 50000}) = 0 (Timeout) gettimeofday({1108935078, 535532}, {360, 0}) = 0 gettimeofday({1108935078, 535816}, {360, 0}) = 0 gettimeofday({1108935078, 536192}, {360, 0}) = 0 write(3, "\31\0\v\0\22\0\200\1\0\0\0\0\37\244\252\267\26\36:\0\22"..., 44) = 44 gettimeofday({1108935078, 536926}, {360, 0}) = 0 write(7, "\0", 1) = 1 futex(0x81c3c50, FUTEX_WAKE, 1) = 1 futex(0x83b5e70, FUTEX_WAKE, 1) = 1 futex(0x83b5e80, FUTEX_WAIT, 492, NULL