Tkinter deadlock on graceful exit

[Matteo Landi]

On Jun/01, Matteo Landi wrote:
> On Thu, May 31, 2012 at 3:02 PM, Matteo Landi <matteo at matteolandi.net> wrote:
> > On Thu, May 31, 2012 at 3:42 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> >> On 5/30/2012 6:19 PM, Matteo Landi wrote:
> >>>
> >>> On May/28, Matteo Landi wrote:
> >>>>
> >>>> Hi list,
> >>>> recently I started to work on an application [1] which makes use of the
> >>>> Tkinter
> >>>> module to handle interaction with the user.  Simply put, the app is a
> >>>> text
> >>>> widget displaying a file filtered by given criteria, with a handy feature
> >>>> that
> >>>> the window is raised each time a new line is added to the widget.
> >>>>
> >>>> The application mainly consists of three threads:  the first one, the
> >>>> file
> >>>> processor, reads the file, filters the lines of interest, and pushes them
> >>>> into
> >>>> a shared queue (henceforth `lines_queue`);  the second one, the
> >>>> gui_updater,
> >>>> pops elements from `lines_queue`, and schedule GUI updates using the
> >>>> `after_idle` method of the Tkinter module;  finally the last one, the
> >>>> worker
> >>>> spawner, receives commands by the gui (by means of a shared queue,
> >>>> `filters_queue`), and drives the application, terminating or spawning new
> >>>> threads.
> >>>>
> >>>> For example, let's see what happens when you start the application, fill
> >>>> the
> >>>> filter entry and press Enter button:
> >>>> 1 the associated even handler is scheduled (we should be inside the
> >>>> Tkinter
> >>>>   mainloop thread), and the filter is pushed into `filters_queue`;
> >>>> 2 the worker spawner receives the new filter, terminate a possibly
> >>>> running
> >>>>   working thread, and once done, create a new file processor;
> >>>> 3 the file processor actually processes the file and fills the
> >>>> `lines_queue`
> >>>>   with the lines matching given filter;
> >>>> 4 the gui updater schedules GUI updates as soon as items are pushed into
> >>>>   `lines_queue`
> >>>> 5 Tkinter mainloop thread updates the gui when idle
> >>>>
> >>>> What happens when the main window is closed?  Here is how I implemented
> >>>> the
> >>>> graceful shutdown of the app:
> >>>> 1 a quit event is scheduled and a _special_ message is pushed into both
> >>>>   `filter_queue` and `lines_queue`
> >>>> 2 the gui updater threads receives the _special_ message, and terminates
> >>>> 3 the worker spawner receives the message, terminates the working thread
> >>>> and
> >>>>   interrupts her execution.
> >>>> 4 Tk.quit() is called after the quit event handler, and we finally quit
> >>>> the
> >>>>   mainloop
> >>>>
> >>>> Honestly speaking, I see no issues with the algorithm presented above;
> >>>>  however,
> >>>> if I close the window in the middle of updates of the text widget, the
> >>>> applications hangs indefinitely.  On the other hand, everything works as
> >>>> expected if I close the app when the file processor, for example, is
> >>>> waiting for
> >>>> new content to filter.
> >>>>
> >>>> I put some logging messages to analyze the deadlock (?!), and noticed
> >>>> that both
> >>>> the worker spawner and the file processor are terminated correctly.  The
> >>>> only
> >>>> thread still active for some strange reasons, is the gui updater.
> >>>>
> >>>> Do you see anything wrong with the description presented above?  Please
> >>>> say so,
> >>>> because I can't figure it out!
> >>
> >>
> >> Since no-one else answered, I will ask some questions based on little
> >> tkinter experience, no thread experience, and a bit of python-list reading.
> >>
> >> 1. Are you only using tkinter in one thread? (It seems like so from the
> >> above)?
> >
> > Yes, provided that `after_idle` queues a job for the gui thread
> >
> >>
> >> 2. Is root.destroy getting called, as in 24.1.2.2. A Simple Hello World
> >> Program in the most recent docs? (I specify 'most recent' because that
> >> example has been recently revised because the previous version sometimes
> >> left tkinter hanging for one of the code paths, perhaps similar to what you
> >> describe.
> >
> > No, I'm not calling the destroy method of the main window but, why
> > that only happens while doing gui updates?
> >
> >>
> >> 3. Have you tried making the gui thread the master thread? (I somehow expect
> >> that the gui thread should be the last to shut down.)
> >
> > No but, same question as above.
> >
> > I'm not home right now, so I will try those solutions as soon as
> > possible.  Thanks.
> >
> >
> > Cheers,
> > Matteo
> >
> >>
> >> --
> >> Terry Jan Reedy
> >>
> >> --
> >> http://mail.python.org/mailman/listinfo/python-list
> >
> >
> >
> > --
> > http://www.matteolandi.net/
> 
> Doing further investigation, I found this post [1] dated 2005, which
> probably explains why I'm gatting this strange deadlock.
> 
> Quoting the linked article:
> 
> All Tkinter access must be from the main thread (or, more precisely,
> the thread that called mainloop). Violating this is likely to cause
> nasty and mysterious symptoms such as freezes or core dumps. Yes this
> makes combining multi-threading and Tkinter very difficult. The only
> fully safe technique I have found is polling (e.g. use after from the
> main loop to poll a threading Queue that your thread writes). I have
> seen it suggested that a thread can safely use event_create to
> communicate with the main thread, but have found this is not safe.
> 
> Well, if this still applies, then I'm invoking `after_idle` from a
> thread different from the mainloop's one.  I'll try to implement the
> poll/queue fix, and see if this solves the issue.
> 
> 
> Matteo
> 
> [1] http://mail.python.org/pipermail/tkinter-discuss/2005-February/000313.html
> -- 
> http://www.matteolandi.net/

I managed to solve the deadlock:  it was a threading problem, and the polling
job + synchronized queue solution [1] worked like a charm.

Lesson learned:  never invoke Tkinter functions / methods outside the mainloop
thread.. NEVER!


Regards,
Matteo

[1] https://bitbucket.org/iamFIREcracker/logfilter/changeset/9e1b571f90eb

-- 
http://www.matteolandi.net