[Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

[Nick Coghlan]

On 11 July 2018 at 00:31, David Foster <davidfstr at gmail.com> wrote:
> I was not aware of PyParallel. The PyParellel "parallel thread"
> line-of-execution implementation is pretty interesting. Trent, big kudos to
> you on that effort.
>
> Since you're speaking in the past tense and said "but we're not doing it
> like that", I infer that the notion of a parallel thread was turned down for
> integration into CPython, as that appears to have been the original goal.
>
> However I am unable to locate a rationale for why that integration was
> turned down. Was it deemed to be too complex to execute, perhaps in the
> context of providing C extension compatibility? Was there a desire to see a
> similar implementation on Linux as well as Windows? Some other reason?

It was never extended beyond Windows, and a Windows-only solution
doesn't meet the needs of a lot of folks interested in more efficient
exploitation of multiple local CPU cores.

It's still an interesting design concept though, especially for
problems that can be deconstructed into a setup phase (read/write main
thread), and a parallel operation phase (ephemeral worker threads that
store all persistent state in memory mapped files, or otherwise
outside the current process).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia