[Python-Dev] PEP 567 v3

[Nathaniel Smith]

On Thu, Jan 18, 2018 at 12:03 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Wed, 17 Jan 2018 20:53:42 -0500
> Yury Selivanov <yselivanov.ml at gmail.com> wrote:
>>
>> Proposed by Antoine Pitrou, this could enable transparent
>> cross-process use of ``Context`` objects, so the
>> `Offloading execution to other threads`_ example would work with
>> a ``ProcessPoolExecutor`` too.
>>
>> Enabling this is problematic because of the following reasons:
>>
>> 1. ``ContextVar`` objects do not have ``__module__`` and
>>    ``__qualname__`` attributes, making straightforward pickling
>>    of ``Context`` objects impossible.  This is solvable by modifying
>>    the API to either auto detect the module where a context variable
>>    is defined, or by adding a new keyword-only "module" parameter
>>    to ``ContextVar`` constructor.
>>
>> 2. Not all context variables refer to picklable objects.  Making a
>>    ``ContextVar`` picklable must be an opt-in.
>
> This is a red herring.  If a value isn't picklable, pickle will simply
> raise as it does in other contexts.  You should't need to opt in for
> anything here.

The complication is that Contexts collect ContextVars from all over
the process. So if people are going to pickle Contexts, we need some
mechanism to make sure that we don't end up in a situation where it
seems to work and users depend on it, and then they import a new
library and suddenly pickling raises an error (because the new library
internally uses a ContextVar that happens not to be pickleable).

-n

-- 
Nathaniel J. Smith -- https://vorpus.org