logging from time critical tasks -- QueueListener().stop() takes the whole CPU

[Gerlando Falauto]

On Monday, July 16, 2018 at 8:13:46 AM UTC+2, Thomas Jollans wrote:
> On 16/07/18 07:39, Gerlando Falauto wrote:
> > On Monday, July 16, 2018 at 6:56:19 AM UTC+2, dieter wrote:
> >>> ...
> >>> Why is the main thread taking up so much CPU?
> >>> I believe at this point listener.stop() should only be waiting for the helper thread to terminate, which I reckon would be implemented by waiting on a semaphore or something (i.e. iowait i.e. 0% CPU).
> >>
> >> Maybe, you look at the source code of "listener.stop"?
> > 
> > I did, forgot to mention that. Culprit is self._thread.join().
> > Which is where it waits for the internal thread to terminate,
> > which I would've expected to wait on a lock or semaphore (pthread_join()?)
> > So there's something I'm totally missing here, which has more to do
> > with queues and threads in general than it has with logging.
> > Any ideas?
> > 
> 
> I have no idea what's really going on there, but a quick look through
> the source reveals that the actual waiting in Thread.join() is done by
> sem_wait().
> 
> via
> https://github.com/python/cpython/blob/13ff24582c99dfb439b1af7295b401415e7eb05b/Python/thread_pthread.h#L304
> via
> https://github.com/python/cpython/blob/master/Modules/_threadmodule.c#L45

Hmm... do you think it's possible it's really getting interrupted the whole time, effectively turning the lock into a sort of spinlock?
Any idea how to confirm that without recompiling the whole code?