[Python-ideas] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

[Adam Olsen]

On 9/19/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 9/18/07, Brian Granger <ellisonbg.net at gmail.com> wrote:
>
> > If there were infinitely many people willing for work on this stuff,
> > then I agree, but I don't see even a dozen people hacking on the GIL.
>
> In part because many people don't believe it would be productive.
>
> For threading to be useful in terms of parallel processing, most
> memory access has to be read-only.  That isn't true today, largely
> because of reference counts.
>
> There are ways around that, by using indirection, or delayed counts,
> or multiple refcount buckets per object, or even just switching to a
> tracing GC.
>
> So far, no one has been able to make these changes without seriously
> mangling the C API and/or slowing things down a lot.  The current
> refcount mechanism is so lightweight that it isn't clear this would
> even be possible.  (With 4 or more cores dedicated to just python, it
> might be worth it anyhow -- but it isn't yet.)  So if you want the GIL
> removed, you need to provide an existence proof that (CPython) memory
> management can be handled efficiently without it.

Is 60-65% of normal CPython "a lot"?

(I really should clean things up and post a patch...)

-- 
Adam Olsen, aka Rhamphoryncus