[Python-ideas] solving multi-core Python

[Sturla Molden]

On 25/06/15 00:19, Eric Snow wrote:

>>> Solving reference counts in this situation is a separate issue that
>>> will likely need to be resolved, regardless of which machinery we use
>>> to isolate task execution.
>>
>> As long as we have a GIL, and we need the GIL to update a reference count,
>> it does not hurt so much as it otherwise would. The GIL hides most of the
>> scalability impact by serializing flow of execution.
>
> It does hurt in COW situations, e.g. forking.  My expectation is that
> we'll at least need to take a serious look into the matter in the
> short term (i.e. Python 3.6).

Yes.

It hurts performance after forking as reference counting will trigger a 
lot of page copies. Keeping reference counts in separate pages and 
replacing the field in the PyObject struct would reduce this problem by 
a factor of up to 512 (64 bit) or 1024 (32 bit).

It does not hurt performance with multi-threading, as Python threads are 
serialized by the GIL. But if the GIL was removed it would result in a 
lot of false sharing. That is a major reason we need a tracing garbage 
collector instead of reference counting if we shall be able to remove 
the GIL.



Sturla