[Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

[Guido van Rossum]

On 9/18/07, Krishna Sankar <ksankar at doubleclix.net> wrote:
> Folks,
>    As a follow-up to the py3k discussions started by Bruce and Guido, I
> pinged Brett and he suggested I submit an exploratory proposal. Would
> appreciate insights, wisdom, the good, the bad and the ugly.
>
> A)    Does it make sense ?
> B)    Which application sets should we consider in designing the
> interfaces and implementations
> C)    In this proposal, parallelism and concurrency are used in an
> interchangeable fashion. Thoughts ?
> D)    Please suggest pertinent links, discussions and insights.
> E)    I have kept the proposal to a minimum to start the discussions and
> to explore if this is the right thing to do. Collaboratively, as we
> zero-in on one or two approaches, the idea is to expand it to a crisp
> and clear PEP. Need to do some more formatting as well.

I'd say it is a little light on specific proposals. The only section
that actually proposes anything is this:

> Pragmatic proposal
> ------------------
> May I suggest we embed two primitives in Python 3K:
> A)    A functional style share-nothing set of interfaces (and
> implementations thereof) - provides  the task parallelism/concurrency
> capability, "small messages, big computations" as Joe Armstrong calls it[3]
> B)    A limited shared memory based model for data intensive parallelism
>
> Most probably this would be part of stdlib. While Guido is almost right
> in saying that this is a (std)library problem, it is not fully so. We
> would need a few primitives from the underlying PVM substrate. Possibly
> one reason for Guido's position is the lack of clarity as to what needs
> to be changed and why. IMHO, just saying take GIL off does not solve the
> problem either.

Before I can meaningfully comment I think I'd like to hear more about
what specifically you are thinking of.

I don't mind the necessary changes to the PVM. I do like to see how
this affects existing C extensions though.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)