[Python-ideas] solving multi-core Python

[Eric Snow]

On Tue, Jun 23, 2015 at 5:32 PM, Devin Jeanpierre
<jeanpierreda at gmail.com> wrote:
> A coworker of mine wrote a patch to Python that allows you to freeze
> refcounts for all existing objects before forking, if the correct
> compile options are set. This adds overhead to incref/decref, but
> dramatically changes the python+fork memory usage story. (I haven't
> personally played with it much, but it sounds decent.) If there's any
> interest I can try to upstream this change, guarded behind a compiler
> flag.
>
> We've also tried moving refcounts to their own pages, like you and
> Nick suggest, but it breaks a *lot* of third-party code. I can try to
> upstream it. If it's guarded by a compiler flag it is probably still
> useful, just any users would have to grep through their dependencies
> to make sure nothing directly accesses the refcount. (The stdlib can
> be made to work.)  It sounds like it would also be useful for the main
> project in the topic of this thread, so I imagine there's more
> momentum behind it.

I'd be interested in more info on both the refcount freezing and the
sepatate refcounts pages.

-eric