Trying to use threading.local()

[Cameron Simpson]

On 13Sep2018 12:21, Antoon Pardon <antoon.pardon at vub.be> wrote:
>On 12-09-18 22:14, Peter Otten wrote:
>> As I understand it you need one local() instance that is shared by all
>> workers. Every thead will then see thread-specific values.
>
>It has always puzzled me how this is useful. The times I work with threads,
>I just put thread specific values in the local variables of the thread function.

That works if your thread function is self contained (only that function needs 
access to the thread specific values) or you can easily pass the thread 
specific values on to subsidiary functions if they're needed.

>What is gained by using threading.local instances?

Supposing you've got some longer lived objects which existed before your thread 
started and survive after it completes, which want to maintain some state for 
client threads, by which I mean your subsequent threads which may make use of 
these objects.

For this purpose the long lived object wants to keep some distinct state for 
each client thread, because they may use it concurrently. Consider an object 
looking after a database, which automatically opens a database connection to 
serve each client thread when it is accessed. You'd make a threading.local 
instance to manage this connection, and each thread would see its own distinct 
connection by accessing that threading.local instance.

There's any number of other examples, but they probably all exist around 
objects essentially external to your thread function which need to provide 
thread specific state which isn't constructed by the thread function itself but 
instead as a consequence of the function accessing the long lived object.

I've got an example here where I maintain a stack of data source contexts for 
some data access. Calling code looks like this:

  with S:
    ... call functions which make use of "S" implicitly ...
    with S2:
      ... here we use "S2" implicitly ...
    ... back to using "S" ...
  ... back to whatever was in use previously ...

The "S" context manager pushes itself onto a stack, and the access code makes 
use of whatever is on the top of that stack. This supports generic code which 
doesn't have to pass "S" to every single function call, including _through_ 
functions which never use "S" but themselve call other functions which do. 

So the current "S" is effectively a piece of global context, like the UNIX 
working directory (it is implicitly used when you open a file with a relative 
pathname).

Now imagine that I have multiple threads working this way. I now need thread 
specific context stacks. I do this with a threading.local instance, with the 
stack an attribute of the threading.local. Multiple threads can now freely push 
and pop from the context stack without interfering with each other.

Hoping this clarifies the use case.

Cheers,
Cameron Simpson <cs at cskk.id.au>