logging from time critical tasks -- QueueListener().stop() takes the whole CPU

[Thomas Jollans]

On 16/07/18 07:39, Gerlando Falauto wrote:
> On Monday, July 16, 2018 at 6:56:19 AM UTC+2, dieter wrote:
>>> ...
>>> Why is the main thread taking up so much CPU?
>>> I believe at this point listener.stop() should only be waiting for the helper thread to terminate, which I reckon would be implemented by waiting on a semaphore or something (i.e. iowait i.e. 0% CPU).
>>
>> Maybe, you look at the source code of "listener.stop"?
> 
> I did, forgot to mention that. Culprit is self._thread.join().
> Which is where it waits for the internal thread to terminate,
> which I would've expected to wait on a lock or semaphore (pthread_join()?)
> So there's something I'm totally missing here, which has more to do
> with queues and threads in general than it has with logging.
> Any ideas?
> 

I have no idea what's really going on there, but a quick look through
the source reveals that the actual waiting in Thread.join() is done by
sem_wait().

via
https://github.com/python/cpython/blob/13ff24582c99dfb439b1af7295b401415e7eb05b/Python/thread_pthread.h#L304
via
https://github.com/python/cpython/blob/master/Modules/_threadmodule.c#L45
via
https://github.com/python/cpython/blob/master/Modules/_threadmodule.c#L136
via https://github.com/python/cpython/blob/master/Lib/threading.py#L1000