[Idle-dev] IDLE freezing/hanging

[John Zelle]

I haven't had a chance yet to try out Kurt's debugging suggestions, but 
I have attached strace. This hang is so bad on my platform (Debian 
testing, 2.6.10 stock [non-debian] kernel, KDE 3.3) as to render IDLE 
unuseable. All I have to do is fire up IDLE and start doing any kind of 
editing on a new file or an existing one, and in very short order, the 
process locks up. Somtimes I am typing, sometimes just when I leave it 
idleing. I have not noticed that typing rate is a factor. The strace 
sequence always ends up the same, I'm attaching one example run. It 
always dies with a FUTEX_WAKE, FUTEX_WAIT, FUTEX_WAKE sequence, with 
strace not printing the complete final call. I have no idea what that 
means. Perhaps someone who knows more can make sense of it.

When I get a chance, I'll run some more experiments along the lines 
suggested below. The really strange part of this is that I had this 
problem before, and _somehow_ managed to get IDLE useable again. I 
thought it was a library upgrade that fixed it. But I now have a clean 
install with all libs up-t0-date, and it's worse than ever.

I am going to try building from the Python source to see if this could 
be a problem with the Debian binary distribution.

--John Zelle

Kurt B. Kaiser wrote:
> Grégoire Dooms <dooms at info.ucl.ac.be> writes:
> 
> 
>>Me too.
>>I use Debian unstable and Idle patched with Raphael Noam's
>>enhancements (they are great I would love to see this merged).
> 
> 
> This will happen fairly soon.  He has an enthusiatic following.
> 
> 
>>I tried to debug the problem but am not aware of a way to attach a
>>debugger to a python process (I would like to do it at a higher level
>>than interpreter level with gdb).
> 
> 
> There is a graphics version of gdb: ddd
> 
> http://www.gnu.org/software/ddd/ddd.html
> 
> 
>>All I could do up to now is strace both processes and see one of them
>>repeating a sequence of futex and select calls (this is normal I think)
>> while the other is blocked in a futex call.
> 
> 
> OK, it may be that the subprocess is somehow blocked looking for input
> from the GUI.  
> 
> 
>>I tried to get that process out of the futex call by sending a signal
>>but all signals I tried up to now made it exit (HUP and CHLD, if I
>>remember correctly). The problem is not easily reproductible as it
>>seems to occur at random. But I feel it could be related to rate of
>>typing as I nearly always get it when typing on my laptop keyboard.
> 
> 
> The blizzard of futex calls is due to polling the sockets.  That is
> done by both the GUI and the subprocess (execution server).  The tricky
> thing debugging IDLE is to do it amid all that activity.
> 
> If you can get to a point where you have something at least partially
> reproducible, try running IDLE with the -n switch :
> 
> (cd ..../Lib/idlelib && ../../python ./PyShell.py -n) or idle -n,
> depending on how your system is set up.  See if the problem goes away.
> 
> 
>>I did not submit a bug as this is patched code and I never had a
>>student annoyed by this bug.
>>I'm willing to help trace and fix the bug if you can direct me in the
>>right direction.
> 
> 
> One thing you can try is to turn on the tracing built into IDLE's 
> rpc.py:
> 
> File / Open Module  : rpc
> 
> In class RPCHandler and class RPCClient there are 'debugging'
> variables.  Set them to True and restart IDLE.  You'll now see
> extensive output in the command window.  Note that this output is an
> interleave of everything going on, and, as is typical of output from
> several threads, may be slightly out of sequence.  But there are
> sequence numbers associated with each rpc call to help you decode all
> that.  If that isn't enough, there are some print>>sys.__stderr__ 
> statements in rpc.py which are commented out.  You can turn them on to
> get a veritable blizzard of traces :-)
> 
> When IDLE freezes, I assume you mean that the GUI is unresponsive, and
> that you can't restart the shell from the Shell menu.  If so, try
> this: kill the subprocess with 
> 
> 'kill -KILL xxxxx'.  The subprocess is the one with '8833' in its
> command line (ps ax).
> 
> IDLE should then spawn a fresh subprocess and you should be able to
> continue without losing any of your work.
> 
> If it doesn't, that points to a problem with Tk.  Does it happen while
> you're typing quickly, or will it freeze while you're having lunch?
> 
> I must say: I've been using IDLE for years, on W98, W2K, WXP, RH6.2,
> Arch Linux, and OpenBSD.  I've never had IDLE freeze on me unless I
> made a coding error while working on IDLE itself.  Then I see a
> traceback in the command window.
> 
> I have had X crash.  And even a couple of emacs hangs.  These things
> seem to happen once in a blue moon when I'm using Gnome on Arch (which
> is pretty bleeding edge).  What's your desktop/window manager?
> 
> (I mostly use Ion, it works extremely well with IDLE because Ion uses
> non-overlapping panes and each pane can contain multiple tabbed IDLE
> windows which can be dragged/dropped into different panes.  After you
> use Ion for awhile you realize that overlapping windows are just an
> awful design choice made at Xerox PARC lo those many years ago.  It's
> also super lightweight and stable.)
> 
> IDLE is written in pure Python.  So if the 'freeze' isn't due to some
> hitherto undetected GUI/subprocess interaction deadlock (which can be
> resolved w/o losing work by killing the subprocess), then there is an
> issue in the C code implementing Python or Tk.  Hardware seems to be
> ruled out because all of you are having problems on Debian Sid.
> 
> And the way to debug that is to attach gdb to it.  It won't be pretty.
> The subprocess has two threads: one executing user code, and one
> minding the sockets. The GUI has one thread which executes the Tk
> event loop, with polling implemented using after() calls.  Additional
> threads are created as necessary for callbacks from the subprocess.
> 
-------------- next part --------------
ioctl(3, FIONREAD, [32])                = 0
read(3, "\36\360\6\v\26\36:\0^\0\0\3\22\0\200\1\1\0\0\0\37\0\0\0"..., 32) = 32
gettimeofday({1108935078, 484691}, {360, 0}) = 0
select(6, [5], [], [], {0, 50000})      = 0 (Timeout)
gettimeofday({1108935078, 535532}, {360, 0}) = 0
gettimeofday({1108935078, 535816}, {360, 0}) = 0
gettimeofday({1108935078, 536192}, {360, 0}) = 0
write(3, "\31\0\v\0\22\0\200\1\0\0\0\0\37\244\252\267\26\36:\0\22"..., 44) = 44
gettimeofday({1108935078, 536926}, {360, 0}) = 0
write(7, "\0", 1)                       = 1
futex(0x81c3c50, FUTEX_WAKE, 1)         = 1
futex(0x83b5e70, FUTEX_WAKE, 1)         = 1
futex(0x83b5e80, FUTEX_WAIT, 492, NULL