[Python-Dev] Timeout for PEP 550 / Execution Context discussion

[Yury Selivanov]

On Mon, Oct 16, 2017 at 11:49 AM, Guido van Rossum <guido at python.org> wrote:
> On Sun, Oct 15, 2017 at 10:26 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> On Sun, Oct 15, 2017 at 10:10 PM, Guido van Rossum <guido at python.org>
>> wrote:
>> > Yes, that's what I meant by "ignoring generators". And I'd like there to
>> > be
>> > a "current context" that's a per-thread MutableMapping with ContextVar
>> > keys.
>> > Maybe there's not much more to it apart from naming the APIs for getting
>> > and
>> > setting it? To be clear, I am fine with this being a specific subtype of
>> > MutableMapping. But I don't see much benefit in making it more abstract
>> > than
>> > that.
>>
>> We don't need it to be abstract (it's fine to have a single concrete
>> mapping type that we always use internally), but I think we do want it
>> to be opaque (instead of exposing the MutableMapping interface, the
>> only way to get/set specific values should be through the ContextVar
>> interface). The advantages are:
>>
>> - This allows C level caching of values in ContextVar objects (in
>> particular, funneling mutations through a limited API makes cache
>> invalidation *much* easier)
>
>
> Well the MutableMapping could still be a proxy or something that invalidates
> the cache when mutated. That's why I said it should be a single concrete
> mapping type. (It also doesn't have to derive from MutableMapping -- it's
> sufficient for it to be a duck type for one, or perhaps some Python-level
> code could `register()` it.

Yeah, we can do a proxy.

>
>>
>> - It gives us flexibility to change the underlying data structure
>> without breaking API, or for different implementations to make
>> different choices -- in particular, it's not clear whether a dict or
>> HAMT is better, and it's not clear whether a regular dict or
>> WeakKeyDict is better.
>
>
> I would keep it simple and supid, but WeakKeyDict is a subtype of
> MutableMapping, and I'm sure we can find a way to implement the full
> MutableMapping interface on top of HAMT as well.

Correct.

>
>>
>> The first point (caching) I think is the really compelling one: in
>> practice decimal and numpy are already using tricky caching code to
>> reduce the overhead of accessing the ThreadState dict, and this gets
>> even trickier with context-local state which has more cache
>> invalidation points, so if we don't do this in the interpreter then it
>> could actually become a blocker for adoption. OTOH it's easy for the
>> interpreter itself to do this caching, and it makes everyone faster.
>
>
> I agree, but I don't see how making the type a subtype (or duck type) of
> MutableMapping prevents any of those strategies. (Maybe you were equating
> MutableMapping with "subtype of dict"?)

Question: why do we want EC objects to be mappings?  I'd rather make
them opaque, which will result in less code and make it more
future-proof.

The key arguments for keeping ContextVar abstraction:

* Naturally avoids name clashes.

* Allows to implement efficient caching.  This is important if we want
libraries like decimal/numpy to start using it.

* Abstracts away the actual implementation of the EC.  This is a
future-proof solution, with which we can enable EC support for
generators in the future.  We already know two possible solutions (PEP
550 v1, PEP 550 current), and ContextVar is a good enough abstraction
to support both of them.

IMO ContextVar.set() and ContextVar.get() is a simple and nice API to
work with the EC.  Most people (aside framework authors) won't even
need to work with EC objects directly anyways.

Yury