Tkinter and cv2: "not responding" popup when imshow launched from tk app

[aapost]

On 3/15/23 07:37, John O'Hagan wrote:
> On Tue, 2023-03-14 at 16:22 -0400, aapost wrote:
>> On 3/14/23 06:54, John O'Hagan wrote:
>>> It works up to a point - I can cycle through the images by clicking
>>> the
>>> button - but if I mouse-click on the displayed image (e.g. to use
>>> the
>>> zooming and panning features of cv2), nothing happens, and a few
>>> seconds later the image greys out and a popup appears saying
>>> "'Unknown'
>>> is not responding" and giving the option of waiting or forcing
>>> close
>>> (but sometimes these options are greyed out too). Clicking "wait",
>>> if
>>> available, closes the popup but it comes back a few seconds later.
>>> If I
>>> then click on the tkinter window titlebar, the popup changes
>>> to "'Tk'
>>> is not responding". Clicking on the button still works and after a
>>> few
>>> clicks the popup closes.
> 
> [...]
> 
>>> I think this particular popup is a Gnome thing, but AIUI most DEs
>>> have
>>> something similar to detect stuck apps. But the app is not stuck.
>>>
>>> I suspect this is some kind of interaction between the call to
>>> cv2.waitKey (which is necessary but I've never understood why!) and
>>> the
>>> tkinter event loop, but it's beyond my knowledge.
>>>
>>> Any suggestions about causes or workarounds?
>>>
>>> Thanks
>>>
>>> --
>>>
>>> John
>>>
>>>
>>
>>
>> I don't get any of the zoom/panning behavior with waitKey(1). But a
>> couple notes from the web:
>>
>>
>> https://docs.opencv.org/2.4/modules/highgui/doc/user_interface.html
>>
>> Note
>>
>> This function should be followed by waitKey function which displays
>> the
>> image for specified milliseconds. Otherwise, it won’t display the
>> image.
>> For example, waitKey(0) will display the window infinitely until any
>> keypress (it is suitable for image display). waitKey(25) will display
>> a
>> frame for 25 ms, after which display will be automatically closed.
>> (If
>> you put it in a loop to read videos, it will display the video
>> frame-by-frame)
>>
>> https://pythonexamples.org/python-opencv-imshow/
>>
>> cv2.waitKey(0) is important for holding the execution of the python
>> program at this statement, so that the image window stays visible. If
>> you do not provide this statement, cv2.imshow() executes in fraction
>> of
>> a second and the program closes all the windows it opened, which
>> makes
>> it almost impossible to see the image on the window.
>>
>> if I change waitKey to 0, I get the ability to zoom/pan, but it
>> places
>> the tk window in a blocked state because it is waiting on cv2 to
>> return.
>> If I hit the ESC key, it releases the wait and gives control back to
>> the
>> tk window, allowing me to press show again to continue to the next
>> image.
>>
>> I can try to change waitKey to a high ms like 10000000 and have
>> zoom/pan
>> for that amount of time before it gives control back to tk (not a
>> sensical approach).
>>
>> The fact that you can still see the image after tk takes back control
>> is
>> I believe just a matter of design, some examples show
>> cv2.destroyAllWindows() after waitKey(0) to clean that up, but of
>> course
>> if you are reusing the same window, that destroys the target.
>>
>> You can resolve that if you move
>> cv2.namedWindow('W', cv2.WND_PROP_FULLSCREEN)
>> cv2.setWindowProperty('W', cv2.WND_PROP_FULLSCREEN,
>> cv2.WINDOW_FULLSCREEN)
>>
>> to inside the show, and destroy it after the wait, and make waitKey
>> 0,
>> this allows creation/cleanup of that window per image
>>
>> Hitting ESC when done zooming/panning on each image to get back to
>> tk.
> 
> Thanks for your reply.
> 
> I'm afraid in the real app it won't be practical to have no gui control
> while viewing images.
> 
> But your suggestions have made me realise that my issue has nothing to
> do with tkinter, it seems to be the way imshow is supposed to work.
> This code gives the same behaviour described above:
> 
> cv2.imshow('W', array)
> cv2.waitKey(100)
> time.sleep(20)
> 
> I've seen the docs you mention that say the window will close if no
> keypress happens within the waitKey time, but that doesn't seem to be
> what happens as you say. After waitKey returns, the image remains and
> everything works, but for some reason the desktop complains about an
> unresponsive app.
> 
> You can use cv2 to display video with:
> 
> while 1:
>   frame = camera.read() #pseudocode
>   cv2.imshow('W', frame)
>   cv2.waitKey(1)
> 
> (In fact this used to work without waitKey if you called
> cv2.startWindowThread() after creating the window, but that stopped
> working at some point.)
> 
> The fact that this works, presumably because the image is being
> replaced often enough, suggested the following workaround to my
> problem, using a call to after() to re-display the current image before
> the desktop starts complaining:
> 
> images=[cv2.imread(i) for i in ('a.jpg', 'b.jpg', 'c.jpg')]
> 
> cv2.namedWindow('W', cv2.WND_PROP_FULLSCREEN)
> cv2.setWindowProperty('W', cv2.WND_PROP_FULLSCREEN,
> cv2.WINDOW_FULLSCREEN)
> 
> counter=[0]
> 
> image = [None]
> 
> def show():
>     im = images[counter[0] % 3]
>     cv2.imshow('W', im)
>     cv2.waitKey(1)
>     image[:] = [im]
>     counter[0] += 1
> 
> root=Tk()
> root.wm_attributes("-topmost", 1)
> b=Button(root, text=' Show ', command=show)
> b.pack()
> 
> def update():
>     if image[0] is not None:
>        cv2.imshow('W', image[0])
>        cv2.waitKey(1)
>        root.after(1000, update)
> 
> update()
> 
> mainloop()
> 
> 
> Not ideal, but I'm starting to think that despite the "highgui" label,
> cv2.imshow is not really suited for integration into a gui. Which is a
> shame because it's so damn fast!



Yep, it was just a solution for the extent of the snippet. I don't have 
any easy fixes for the single thread blocking issue as I have not needed 
to investigate any UI blocking situations yet (if anyone has experience 
in that they can chime in, or you may need to ask about 
blocking/threading solutions/examples in regard to imshow on a cv2 
specific forum, some sort of entirely different approach may be needed).

To clarify the docs, it does not say "close if no key presses", it says 
2 different things. 2nd part being, Setting the ms will close the window 
after the time regardless. The wording of the "displaying the window 
indefinitely until any keypress" - when 0, does not say anything about 
closing, so the until a keypress is unclear as to what that means, as 
any keypress when it is 0 definitely does not close it. They do activate 
the pan/zoom, and ability to ESC though..

Also to clarify (just in case), waitKey is in ms, and time.sleep is in 
seconds, so waitKey 100 is 0.1, the equivalent to time.sleep(20) would 
be waitKey(20000)

The likely reason the desktop is complaining about an unresponsive app 
is that even though you still see that image, even at 100 ms (0.1s) the 
window you are clicking on by that time is technically already dead and 
closed, just not cleaned up.

Additionally at least one of the error messages you were getting were in 
regard to qt (one of the graphical options under the hood), my 
understanding is that qt has some added controls for imshow that show up 
in a toolbar above the image (I don't have them so I don't see them). A 
sort of hypothetical guess, but it might be that those features are 
getting launched, but when they don't have the live imshow to operate 
on, they choke and the OS steps in (could be wrong).

Does the update solve your problem in practice though? That is just a 
loop that makes the imshow live again for 1 ms, then pauses for 1 
second, then runs recursively. Potentially indefinitely delaying 
anything from being flagged as stuck. I assume that doesn't give you the 
pan/zoom ability either (at least it doesn't on mine).