[Python-ideas] solving multi-core Python

[Sturla Molden]

Nick Coghlan <ncoghlan at gmail.com> wrote:

> Sturla, expressing out and out contempt for entire communities of
> capable, competent developers (both the creators of Windows and Java,
> and the users of those platforms) has no place on the core Python
> mailing lists. Please refrain from casually insulting entire groups of
> people merely because you don't approve of their technical choices.

I am not sure what you mean. Using threads on Windows and Java comes from a
necessity, not because developers are incompetent. Windows does not provide
a fork and processes are heavy-weight, hence multi-threading is the obvious
choice.


> Avoiding object serialisation is indeed the main objective. 

Good.


> With
> subinterpreters, we have a lot more options for that than we do with
> any form of IPC, including shared references to immutable objects, and
> the PEP 3118 buffer API.

Perhaps. One could do this with shared memory as well, but a complicating
factor is that the base address must be the same (or corrected for). But
one could probably do low-level magic with memory mapping  to work around
this. Particularly on 64-bit it is not really difficult to make sure a page
is mapped to the same address in two processes.

It is certainly easier to achieve within a process. But if the plan for
Erlang-style "share nothing" threads is to pickle and memcpy objects, there
is little or nothing to gain over using multiprocessing.


Sturla