Tkinter deadlock on graceful exit

[Matteo Landi]

On Thu, May 31, 2012 at 3:02 PM, Matteo Landi <matteo at matteolandi.net> wrote:
> On Thu, May 31, 2012 at 3:42 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>> On 5/30/2012 6:19 PM, Matteo Landi wrote:
>>>
>>> On May/28, Matteo Landi wrote:
>>>>
>>>> Hi list,
>>>> recently I started to work on an application [1] which makes use of the
>>>> Tkinter
>>>> module to handle interaction with the user.  Simply put, the app is a
>>>> text
>>>> widget displaying a file filtered by given criteria, with a handy feature
>>>> that
>>>> the window is raised each time a new line is added to the widget.
>>>>
>>>> The application mainly consists of three threads:  the first one, the
>>>> file
>>>> processor, reads the file, filters the lines of interest, and pushes them
>>>> into
>>>> a shared queue (henceforth `lines_queue`);  the second one, the
>>>> gui_updater,
>>>> pops elements from `lines_queue`, and schedule GUI updates using the
>>>> `after_idle` method of the Tkinter module;  finally the last one, the
>>>> worker
>>>> spawner, receives commands by the gui (by means of a shared queue,
>>>> `filters_queue`), and drives the application, terminating or spawning new
>>>> threads.
>>>>
>>>> For example, let's see what happens when you start the application, fill
>>>> the
>>>> filter entry and press Enter button:
>>>> 1 the associated even handler is scheduled (we should be inside the
>>>> Tkinter
>>>>   mainloop thread), and the filter is pushed into `filters_queue`;
>>>> 2 the worker spawner receives the new filter, terminate a possibly
>>>> running
>>>>   working thread, and once done, create a new file processor;
>>>> 3 the file processor actually processes the file and fills the
>>>> `lines_queue`
>>>>   with the lines matching given filter;
>>>> 4 the gui updater schedules GUI updates as soon as items are pushed into
>>>>   `lines_queue`
>>>> 5 Tkinter mainloop thread updates the gui when idle
>>>>
>>>> What happens when the main window is closed?  Here is how I implemented
>>>> the
>>>> graceful shutdown of the app:
>>>> 1 a quit event is scheduled and a _special_ message is pushed into both
>>>>   `filter_queue` and `lines_queue`
>>>> 2 the gui updater threads receives the _special_ message, and terminates
>>>> 3 the worker spawner receives the message, terminates the working thread
>>>> and
>>>>   interrupts her execution.
>>>> 4 Tk.quit() is called after the quit event handler, and we finally quit
>>>> the
>>>>   mainloop
>>>>
>>>> Honestly speaking, I see no issues with the algorithm presented above;
>>>>  however,
>>>> if I close the window in the middle of updates of the text widget, the
>>>> applications hangs indefinitely.  On the other hand, everything works as
>>>> expected if I close the app when the file processor, for example, is
>>>> waiting for
>>>> new content to filter.
>>>>
>>>> I put some logging messages to analyze the deadlock (?!), and noticed
>>>> that both
>>>> the worker spawner and the file processor are terminated correctly.  The
>>>> only
>>>> thread still active for some strange reasons, is the gui updater.
>>>>
>>>> Do you see anything wrong with the description presented above?  Please
>>>> say so,
>>>> because I can't figure it out!
>>
>>
>> Since no-one else answered, I will ask some questions based on little
>> tkinter experience, no thread experience, and a bit of python-list reading.
>>
>> 1. Are you only using tkinter in one thread? (It seems like so from the
>> above)?
>
> Yes, provided that `after_idle` queues a job for the gui thread
>
>>
>> 2. Is root.destroy getting called, as in 24.1.2.2. A Simple Hello World
>> Program in the most recent docs? (I specify 'most recent' because that
>> example has been recently revised because the previous version sometimes
>> left tkinter hanging for one of the code paths, perhaps similar to what you
>> describe.
>
> No, I'm not calling the destroy method of the main window but, why
> that only happens while doing gui updates?
>
>>
>> 3. Have you tried making the gui thread the master thread? (I somehow expect
>> that the gui thread should be the last to shut down.)
>
> No but, same question as above.
>
> I'm not home right now, so I will try those solutions as soon as
> possible.  Thanks.
>
>
> Cheers,
> Matteo
>
>>
>> --
>> Terry Jan Reedy
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>
>
>
> --
> http://www.matteolandi.net/

Doing further investigation, I found this post [1] dated 2005, which
probably explains why I'm gatting this strange deadlock.

Quoting the linked article:

All Tkinter access must be from the main thread (or, more precisely,
the thread that called mainloop). Violating this is likely to cause
nasty and mysterious symptoms such as freezes or core dumps. Yes this
makes combining multi-threading and Tkinter very difficult. The only
fully safe technique I have found is polling (e.g. use after from the
main loop to poll a threading Queue that your thread writes). I have
seen it suggested that a thread can safely use event_create to
communicate with the main thread, but have found this is not safe.

Well, if this still applies, then I'm invoking `after_idle` from a
thread different from the mainloop's one.  I'll try to implement the
poll/queue fix, and see if this solves the issue.


Matteo

[1] http://mail.python.org/pipermail/tkinter-discuss/2005-February/000313.html
-- 
http://www.matteolandi.net/