[Python-Dev] bpo-34837: Multiprocessing.Pool API Extension - Pass Data to Workers w/o Globals

[Michael Selik]

On Thu, Oct 18, 2018 at 8:35 AM Sean Harrington <seanharr11 at gmail.com>
wrote:

> The most common use case comes up when passing instance methods (of really
> big objects!) to Pool.map().
>

This reminds me of that old joke: "A patient says to the doctor, 'Doctor,
it hurts when I ...!' The doctor replies, 'Well, don't do that.'"

Further, let me pivot on my idea of __qualname__...we can use the `id` of
> `func` as the cache key to address your concern, and store this `id` on the
> `task` tuple (i.e. an integer in-lieu of the `func` previously stored
> there).
>

Possible. Does the Pool keep a reference to the passed function in the main
process? If not, couldn't the garbage collector free that memory location
and a new function could replace it? Then it could have the same qualname
and id in CPython. Edge case, for sure. Worse, it'd be hard to reproduce as
it'd be dependent on the vagaries of memory allocation.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20181018/d98e0307/attachment.html>