[Python-Dev] PEP 567 pre v3

[Nathaniel Smith]

On Tue, Jan 9, 2018 at 3:41 AM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> On Tue, Jan 9, 2018 at 11:02 AM, Nathaniel Smith <njs at pobox.com> wrote:
>> Right now, the set of valid states for a ContextVar are: it can hold
>> any Python object, or it can be undefined. However, the only way it
>> can be in the "undefined" state is in a new Context where it has never
>> had a value; once it leaves the undefined state, it can never return
>> to it.
>
> Is "undefined" a state when a context variable doesn't have a default
> and isn't yet set?  If so, why can't it be returned back to the
> "undefined" state?  That's why we have the 'reset' method:

Sorry, yes, you can return to the "undefined" state if you have a
valid token that hasn't been used yet. But my point is that it's weird
to have a variable that needs so many words to describe which kinds of
state transitions are possible.

> I don't like how context variables are defined in Option 1 and Option
> 2.  I view ContextVars as keys in some global context mapping--akin to
> Python variables.

This is one totally reasonable option but, I mean... it's software,
there are lots of options for how to view things that we could
potentially make true, and we get to pick the one that works best :-).
And I think it's easier to explain to users how ContextVar works if we
can do it without talking about Context mappings etc.

Thread-local storage is also implemented using some per-thread maps
and various clever tricks, but I don't think I've ever seen
documentation that described it that way.

> In any case, at this point I think that the best option is to simply
> drop the "default" parameter from the ContextVar constructor.  This
> would leave us with only one default in ContextVar.get() method:
>
>     c.get()   # Will raise a LookupError if 'c' is not set
>     c.get('python')  # Will return 'python' if 'c' is not set
>
> I also now see how having two different 'default' values: one defined
> when a ContextVar is created, and one can be passed to
> ContextVar.get() is confusing.

But the constructor default is way more important for usability than
any of the other features we're talking about! Every time I use
threading.local, I get annoyed that there isn't a simpler way to
specify a default value. OTOH I've never found a case where I actually
wanted undefined values.

To find out whether my experience is typical, I did a quick grep of
the stdlib, and found 5 thread local variables:

- asyncio.events._BaseEventLoopPolicy._local.{_loop, _set_called}:
These two use the 'subclass threading.local' trick to make it seem
like these are initialized to None.

- asyncio.events._running_loop: This uses the subclass trick to to
make it seem like it's initialized to (None, None).

- multiprocessing.context._tls.spawning_popen: Here the code defines
two accessors (get_spawning_popen, set_spawning_popen) that make it
seem like it's initialized to None. Of course you could do this with
ContextVar's too, but if ContextVar had native support for specifying
an initial value then they'd be unnecessary, because
spawning_popen.get()/set() would already do the right thing.

- _pydecimal.local.__decimal_context__: This is a little trickier. It
has a default value, but it's mutable, so access is hidden behind two
accessors (getcontext, setcontext) and it's initialized on first
access. Currently this is done with a try: except:, but if thread
locals had the ability to set the default to None, then using that
would make the implementation shorter and faster (exceptions are
expensive).

So that's 5 out of 5 cases where the code would get simpler if
thread-locals had the ability to specify a default initial value, 0
out of 5 cases where anyone would miss support for undefined values or
wants to be able to control the default get() value on a call-by-call
basis.

The argument for supporting undefined values in ContextVar is mostly
by analogy with regular Python variables. I like analogies, but I
don't think we should sacrifice actual use cases to preserve the
analogy.

> But I'd be -1 on making all ContextVars have a None default
> (effectively have a "ContextVar.get(default=None)" signature. This
> would be a very loose semantics in my opinion.

It may have gotten lost in that email, but my actual favorite approach
is that we make the signatures:

ContextVar(name, *, initial_value)  # or even (*, name, initial_value)
ContextVar.get()
ContextVar.set(value)

so that when you create a ContextVar you always state the initial
value, whatever makes sense in a particular case. (Obviously None will
be a very popular choice, but this way it won't be implicit, and
no-one will be surprised to see it returned from get().)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org