[Python-Dev] PEP 550 leak-in vs leak-out, why not just a ChainMap

[Yury Selivanov]

Hi Jim,

Sorry, I don't answer all questions/points directly.  We are working
on a new version of the PEP that will hopefully address most of them.

Some comments inlined below:

On Thu, Aug 24, 2017 at 12:32 AM, Jim J. Jewett <jimjjewett at gmail.com> wrote:
[..]
> I still don't see how this is different from a ChainMap.

Because conceptually it's not different.  In the next version of the
PEP it will become more apparent.

[..]
>
>> HAMT is a way to efficiently implement immutable mappings ...
>> using regular dicts and copy, set() would be O(log N)

The key requirement for using immutable datastructures is to make
"get_execution_context" operation fast.  Currently, the PEP doesn't do
a good job at explaining why we need that operation and why it will be
used by asyncio.Task and call_soon, so I understand the confusion.
This will be fixed in the next PEP version.

>
> Using a ChainMap, set affects only the localmost map and is therefore
> O(1).  get could require stacksize lookups, but ...

Again, the PEP essentially is implemented through a very specialized
version of ChainMap, although written in C with a few extra
optimizations and properties.

>
> (A)  How many values do you expect a typical generator to use?  The
> django survey suggested mostly 0, sometimes 1, occasionally 2.  So
> caching the values of all possible keys probably won't pay off.

Not many, but caching is still as important, because some API users
want the "get()" operation to be as fast as possible under all
conditions.

>
> (B)  Other than the truly global context and thread-level context, how
> many of these maps do you expect to be non-empty?

It's likely that most of them will be empty in most cases. Although we
can imagine a degenerate case of a recursive generator that modifies
EC on each iteration.

>
> (C)  How deep do you expect the stack to get?  Are we talking about
> 100 layers of mappings to check between the current generator and the
> thread-wide defaults?  Even if we are, verifying that there hasn't
> been a change in some mid-level layer requires tracking the versions
> of each mid-level layer.  (If version is globally unique, that would
> also ensure that the stack hasn't changed.)  Is that really faster
> than checking that the map is empty?

The stack can be as deep as recursion limit.

>
> And, of course, using a ChainMap means that the keys do NOT have to be
> predefined ... so the Key class really can be skipped.

The first version of the PEP had no ContextKey object and the most
popular complaint about it was that the key names will clash.

The ContextKey design avoids clashing issue entirely and enables fast
Python and C APIs (because of the cache).

Yury