[Patches] [ python-Patches-1252236 ] Simplying Tkinter's event loop

[SourceForge.net]

Patches item #1252236, was opened at 2005-08-04 17:51
Message generated for change (Comment added) made by jimjjewett
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1252236&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Tkinter
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Submitted By: Michiel de Hoon (mdehoon)
Assigned to: Martin v. Löwis (loewis)
Summary: Simplying Tkinter's event loop

Initial Comment:
This patch is a simplification of Tkinter's event loop
by moving the event loop to Python itself.

Tkinter needs an event loop to be able to handle both
Tcl/Tk events and keyboard input by the user. Usually,
an event loop looks like this:

while true:
    wait for an event (Tcl/Tk events or keyboard input)
    handle the event

Tkinter's loop is set up differently. If Tkinter is
imported, it sets the PyOS_InputHook pointer to the
EventHook function in _tkinter.c. The EventHook
function handles Tcl/Tk events only. In essence,
Tkinter's event loop looks like this:

while true:
    handle all Tcl/Tk events that need to be handled
    if keyboard input is waiting: break
    sleep for 20 milliseconds

Note that the event loop exits only if the user presses
a key.
This causes the following problems:
1) This event loop is available only for Tkinter. There
is no way to handle events other than the Tcl/Tk events.
2) If another extension module needs events to be
handled, it can do so by using a similar event loop.
However, it is not possible to handle both Tcl/Tk
events and other events, (except for keyboard input).
3) Chaining the two event loops will fail. As soon as
Tkinter's event loop starts running, it will break out
of this loop only in the case of keyboard input. So any
other events will not be processed until a key is pressed.
4) Even if Tkinter's event loop is the only event loop
that is needed, it can fail. If a Python thread is
waiting for a mutex variable to change, it cannot run
Tkinter's event loop while waiting, since Tkinter's
event loop will only return after a key is pressed (it
doesn't pay attention to the mutex variable). This is
the reason that running Tkinter in an IDLE shell will fail.
5) On some platforms (e.g. Cygwin), the event loop is
unable to check for keyboard input. Hence, one needs to
call mainloop() in order to see Tkinter's widgets.

The reason that the event loop was set up in this
peculiar way  may be due to a bug with PyOS_InputHook
in Python. If Python is compiled with readline support,
PyOS_InputHook will be called ten times per second,
while Python is waiting for keyboard input. (As shown
below, this allows us to fix this event loop problem.)
However, if Python is compiled without readline support
(notably on Windows), PyOS_InputHook is called only
once (just before a user starts entering a new command
on the keyboard). This necessitates Tkinter to run an
event loop, and not return from it until keyboard input
is ready.

If PyOS_InputHook is called ten times per second, we
can set up the event loop as follows:

Set PyOS_InputHook to Tkinter's EventHook

while true:
    call PyOS_InputHook
    handle keyboard input, if present
    sleep for 100 milliseconds

Here, the key point is that Tkinter's EventHook returns
immediately if there are no more Tcl/Tk events to be
handled. Effectively, we have moved the event loop from
Tkinter to Python itself.

Hence, there are two problems to be solved:
1) PyOS_InputHook should be called ten times per
second, regardless of whether Python is compiled with
or without readline support.
2) Tkinter's EventHook should return immediately after
handling all Tcl/Tk events.

Problem 1) is solved in patch #1049855; problem 2) is
solved in this patch. This patch is a considerable
simplication of _tkinter.c, as it removes all lines
that are no longer needed (and adds almost no code).

This patch changes the nature of the function to which
PyOS_InputHook points. Before, PyOS_InputHook is used
to start a Tcl/Tk event loop. Now, PyOS_InputHook is
called from an existing event loop to handle Tcl/Tk events.

With these two patches, Python has a functioning
PyOS_InputHook, which can be used both by Tkinter and
by other extension modules. We can handle Tcl/Tk events
by calling Tkinter's EventHook function if we're
waiting for something else other than keyboard input,
like a mutex variable. We can chain event-handling
functions from different extension modules (e.g.
Tkinter and some other module), by first calling
Tkinter's EventHook and then the other extension
module's event hook function.
Finally, on Cygwin, we can now create Tkinter widgets
without having to call mainloop. So this will now work
on Cygwin:

>>> from Tkinter import *
>>> Label()
>>> # label appears, without having to call mainloop.

This patch has been tested on WIndows and Cygwin.

----------------------------------------------------------------------

Comment By: Jim Jewett (jimjjewett)
Date: 2005-08-15 09:35

Message:
Logged In: YES 
user_id=764593

Clarifying (1a) -- Why (pre-patch) were Windows and 
Unix intentionally set to act differently?  Is there something 
in the default runtimes or libraries which makes 
(expected?) performance very different on the platforms?

Clarifying (1b) -- Is ten times per second enough?  0.1 
seconds is long enough that people can notice it.  If the 
pre-patch version cycles 50 times/second, then going to 
only 10 times/second might make the interface seem 
sluggish.  I'm not sure I'm qualified to make this 
judgement myself, but it is a concern.

Clarifying (1c) -- My (possibly wrong) understanding is 
that before this pair of patches, unix effectively did an 
active check for input, but the windows version waited for 
notification from the OS that keyboard input was available 
-- and after this, both would actively check N times/
second.  If both are just passively waiting, then I'm not 
sure what the 20ms/100ms timeout actually does.

If python is running as a batch process, then forcing it 
back into the "is there input" section several times a 
second (even though there is never any keyboard input) 
will cause the program to take more clocktime.

Clarifying (2) -- The pre-patch version can certainly take 
events (including keyboard events) during the event loop.  
What you can't do is:

"""
(define/run a bunch of stuff)
...
start the event loop
...
(define/run a bunch more stuff)
"""

You need to set up all the definitions and event handlers 
before the loop starts, or else do them as a result of 
events.  Roughly, calling mainloop has to be the *last* 
thing you do in a given thread.  Which leads to (3)

Clarifying (3)  -- Why not just assume threads?  The 
problem you are trying to solve can't exist without threads. 
 Assuming threads won't make anything fail any harder 
than it does now.  If you default to dummy-threads and 
ensure that the event-loop the *last* pseudo-thread, you'll 
even clear up some bugs in carelessly written single-
threaded code. 


----------------------------------------------------------------------

Comment By: Michiel de Hoon (mdehoon)
Date: 2005-08-15 00:46

Message:
Logged In: YES 
user_id=488897

> (1)  Why was the behavior different before?
Actually, the behavior is not much different from before;
this patch together with patch #1049855 essentially move the
event loop from Tkinter to Python core, so that it is
available to other extension modules also. But effectively
the program goes through the same steps as before.
If you want to know why the design of Tkinter's event loop
is the way it is: I am not quite sure, but it may just be a
quick solution to get Tkinter working (and it does work fine
as long as you're interested in Tkinter only). Since Tcl/Tk
already has an event loop, it is easy to run that event loop
and let it check for keyboard input (via
Tcl_CreateFileHandler in EventHook in _tkinter.c). Writing
such a loop for Python is not extremely difficult but also
not straightforward (see my woes with patch #1049855).

> Is 10 times per second not responsive enough?
With the current Python, the loop sleeps every 20 ms before
checking for keyboard input and handling Tcl/Tk events. With
the patch, Tcl/Tk events are handled every 100 ms; keyboard
input is handled much faster (depending on how quickly
"select" on Unix or  "WaitForSingleObject" on Windows
respond to keyboard input). Of course, these delays can be
set to some other value in the code.

> Does a busy-wait of 10 times per second cause too much
thrashing?
I am not sure if I understand this question correctly. The
"select" function on Unix and "WaitForSingleObject" function
on Windows do not do a busy-wait.

> (2)  It seems like the problem isn't really about Tkinter so 
> much as it is about event loops vs threading. The event 
> loop is infinite, so nothing else *in that thread* will
happen 
> after it.  This isn't solvable with a single-threaded
python.  

Sure it's solvable. Even the current implementation of
Tkinter can handle Tcl/Tk events as well as listen for
keyboard input. If you import Tkinter and create a Tk
widget, you can still issue Python commands and move the Tk
widget around, even though they are running in the same
thread. The problem with the current implementation is that
it works for Tkinter only, and secondly, that it doesn't
allow chaining of hook functions. Patch #1049855 solves this
by calling select on Unix (WaitForSingleObject on Windows)
to find out if keyboard input is available, and if not, to
handle Tk/Tcl events. No need for a separate thread for that.

> (On the other hand, single-threaded python should never 
> have the mutex problem you mentioned.)


> (3)  With multi-threaded python, is there any reason not to 
> start the event loop in a fresh thread?  (And let that new 
> thread block waiting for events.)  This would also reduce 
> contention with other frameworks that want to treat the 
> "main" thread differently.

Yes, this is possible on multi-threaded python. However, an
extension module (such as Tkinter) cannot rely on the
assumption that Python is multi-threaded. Personally, I am
interested in PyOS_InputHook for scientific visualization
software, which is likely to be used on all kinds of
outlandish systems. I don't know if I can expect a
multi-threaded Python to be available on all those systems.
Secondly, given that Python has the PyOS_InputHook
functionality, why not make sure it is implemented
correctly? Meaning that its behavior should not depend on
whether readline is installed or not, and that its usage in
Tkinter does not block other extension modules from using it.

I am not sure if I interpreted all of your questions
correctly. Please let me know if I didn't.



----------------------------------------------------------------------

Comment By: Jim Jewett (jimjjewett)
Date: 2005-08-12 11:29

Message:
Logged In: YES 
user_id=764593

(1)  Why was the behavior different before?  Is 10 times 
per second not responsive enough?  Does a busy-wait of 
10 times per second cause too much thrashing?

(2)  It seems like the problem isn't really about Tkinter so 
much as it is about event loops vs threading.  The event 
loop is infinite, so nothing else *in that thread* will happen 
after it.  This isn't solvable with a single-threaded python.  
(On the other hand, single-threaded python should never 
have the mutex problem you mentioned.)

(3)  With multi-threaded python, is there any reason not to 
start the event loop in a fresh thread?  (And let that new 
thread block waiting for events.)  This would also reduce 
contention with other frameworks that want to treat the 
"main" thread differently.



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1252236&group_id=5470