[Python-Dev] PEP 567 pre v3

[Chris Jerdonek]

On Mon, Jan 8, 2018 at 11:02 PM, Nathaniel Smith <njs at pobox.com> wrote:
> Right now, the set of valid states for a ContextVar are: it can hold
> any Python object, or it can be undefined. However, the only way it
> can be in the "undefined" state is in a new Context where it has never
> had a value; once it leaves the undefined state, it can never return
> to it.

I know Yury responded to one aspect of this point later on in the
thread. However, in terms of describing the possible states without
reference to the internal Context mappings, IIUC, wouldn't it be more
accurate to view a ContextVar as a stack of values rather than just
the binary "holding an object or not"? This is to reflect the number
of times set() has been called (and so the number of times reset()
would need to be called to "empty" the ContextVar).

--Chris



>
> This makes me itch. It's very weird to have a mutable variable with a
> valid state that you can't reach by mutating it. I see two
> self-consistent ways to make me stop itching: (a) double-down on
> undefined as being part of ContextVar's domain, or (b) reduce the
> domain so that undefined is never a valid state.
>
> # Option 1
>
> In the first approach, we conceptualize ContextVar as being a
> container that either holds a value or is empty (and then there's one
> of these containers for each context). We also want to be able to
> define an initial value that the container takes on when a new context
> materializes, because that's really convenient. And then after that we
> provide ways to get the value (if present), or control the value
> (either set it to a particular value or unset it). So something like:
>
> var1 = ContextVar("var1")  # no initial value
> var2 = ContextVar("var2", initial_value="hello")
>
> with assert_raises(SomeError):
>     var1.get()
> # get's default lets us give a different outcome in cases where it
> would otherwise raise
> assert var1.get(None) is None
> assert var2.get() == "hello"
> # If get() doesn't raise, then the argument is ignored
> assert var2.get(None) == "hello"
>
> # We can set to arbitrary values
> for var in [var1, var2]:
>     var.set("new value")
>     assert var.get() == "new value"
>
> # We can unset again, so get() will raise
> for var in [var1, var2]:
>     var.unset()
>     with assert_raises(SomeError):
>         var.get()
>     assert var.get(None) is None
>
> To fulfill all that, we need an implementation like:
>
> MISSING = make_sentinel()
>
> class ContextVar:
>     def __init__(self, name, *, initial_value=MISSING):
>         self.name = name
>         self.initial_value = initial_value
>
>     def set(self, value):
>         if value is MISSING: raise TypeError
>         current_context()._dict[self] = value
>         # Token handling elided because it's orthogonal to this issue
>         return Token(...)
>
>     def unset(self):
>         current_context()._dict[self] = MISSING
>         # Token handling elided because it's orthogonal to this issue
>         return Token(...)
>
>     def get(self, default=_NOT_GIVEN):
>         value = current_context().get(self, self.initial_value)
>         if value is MISSING:
>             if default is _NOT_GIVEN:
>                 raise ...
>             else:
>                 return default
>         else:
>             return value
>
> Note that the implementation here is somewhat tricky and non-obvious.
> In particular, to preserve the illusion of a simple container with an
> optional initial value, we have to encode a logically undefined
> ContextVar as one that has Context[var] set to MISSING, and a missing
> entry in Context encodes the presence of the inital value. If we
> defined unset() as 'del current_context._dict[self]', then we'd have:
>
> var2.unset()
> assert var2.get() is None
>
> which would be very surprising to users who just want to think about
> ContextVars and ignore all that stuff about Contexts. This, in turn,
> means that we need to expose the MISSING sentinel in general, because
> anyone introspecting Context objects directly needs to know how to
> recognize this magic value to interpret things correctly.
>
> AFAICT this is the minimum complexity required to get a complete and
> internally-consistent set of operations for a ContextVar that's
> conceptualized as being a container that either holds an arbitrary
> value or is empty.
>
> # Option 2
>
> The other complete and coherent conceptualization I see is to say that
> a ContextVar always holds a value. If we eliminate the "unset" state
> entirely, then there's no "missing unset method" -- there just isn't
> any concept of an unset value in the first place, so there's nothing
> to miss. This idea shows up in lots of types in Python, actually --
> e.g. for any exception object, obj.__context__ is always defined. Its
> value might be None, but it has a value. In this approach,
> ContextVar's are similar.
>
> To fulfill all that, we need an implementation like:
>
> class ContextVar:
>     # Or maybe it'd be better to make initial_value mandatory, like this?
>     #     def __init__(self, name, *, initial_value):
>     def __init__(self, name, *, initial_value=None):
>         self.name = name
>         self.initial_value = initial_value
>
>     def set(self, value):
>         current_context()._dict[self] = value
>         # Token handling elided because it's orthogonal to this issue
>         return Token(...)
>
>     def get(self):
>         return current_context().get(self, self.initial_value)
>
> This is also a complete and internally consistent set of operations,
> but this time for a somewhat different way of conceptualizing
> ContextVar.
>
> Actually, the more I think about it, the more I think that if we take
> this approach and say that every ContextVar always has a value, it
> makes sense to make initial_value= a mandatory argument instead of
> defaulting it to None. Then the typing works too, right? Something
> like:
>
> ContextVar(name: str, *, initial_value: T) -> ContextVar[T]
> ContextVar.get() -> T
> ContextVar.set(T) -> Token
>
> ? And it's hardly a burden on users to type 'ContextVar("myvar",
> initial_value=None)' if that's what they want.
>
> Anyway... between these two options, I like Option 2 better because
> it's substantially simpler without (AFAICT) any meaningful reduction
> in usability. But I'd prefer either of them to the current PEP 567,
> which seems like an internally-contradictory hybrid of these ideas. It
> makes sense if you know how the code and Contexts work. But if I was
> talking to someone who wanted to ignore those details and just use a
> ContextVar, and they asked me for a one sentence summary of how it
> worked, I wouldn't know what to tell them.
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.jerdonek%40gmail.com