Is there a more efficient threading lock?

[Thomas Passin]

On 2/25/2023 10:52 AM, Skip Montanaro wrote:
> I have a multi-threaded program which calls out to a non-thread-safe
> library (not mine) in a couple places. I guard against multiple
> threads executing code there using threading.Lock. The code is
> straightforward:
> 
> from threading import Lock
> 
> # Something in textblob and/or nltk doesn't play nice with no-gil, so just
> # serialize all blobby accesses.
> BLOB_LOCK = Lock()
> 
> def get_terms(text):
>      with BLOB_LOCK:
>          phrases = TextBlob(text, np_extractor=EXTRACTOR).noun_phrases
>      for phrase in phrases:
>          yield phrase
> 
> When I monitor the application using py-spy, that with statement is
> consuming huge amounts of CPU. Does threading.Lock.acquire() sleep
> anywhere? I didn't see anything obvious poking around in the C code
> which implements this stuff. I'm no expert though, so could easily
> have missed something.

I'm no expert on locks, but you don't usually want to keep a lock while 
some long-running computation goes on.  You want the computation to be 
done by a separate thread, put its results somewhere, and then notify 
the choreographing thread that the result is ready.

This link may be helpful -

https://anandology.com/blog/using-iterators-and-generators/