[Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

[Steven H. Rogers]

Krishna Sankar wrote:
> Folks,
>    As a follow-up to the py3k discussions started by Bruce and Guido, I 
> pinged Brett and he suggested I submit an exploratory proposal. Would 
> appreciate insights, wisdom, the good, the bad and the ugly.
>
> A)    Does it make sense ?
> B)    Which application sets should we consider in designing the 
> interfaces and implementations
> C)    In this proposal, parallelism and concurrency are used in an 
> interchangeable fashion. Thoughts ?
> D)    Please suggest pertinent links, discussions and insights.
> E)    I have kept the proposal to a minimum to start the discussions and 
> to explore if this is the right thing to do. Collaboratively, as we 
> zero-in on one or two approaches, the idea is to expand it to a crisp 
> and clear PEP. Need to do some more formatting as well.
> Cheers
> <k/>
> P.S : I had sent this to python-ideas couple of days ago and received 
> two comments (Thanks Leonardo, Thanks Adam) I haven't incorporated their 
> comments yet. Folks who are on both lists, pardon me for the spam.
# Proto-PEP elided.

Other than number of cores, you don't mention hardware architecture.  I 
presume that you're thinking of symmetric multiprocessor architectures.  
If so, this should be explicit.  You should also consider that SMP may 
not be the predominant multi-core architecture in the future, the Cell 
processor has one general purpose processor and eight more specialized 
processors.  You might not want to limit the PEP to 32 cores, I know of 
startups working on 40 and 64 core chips.

Shared memory may be necessary for good performance, but it doesn't have 
to be exposed at the language level.  While Erlang has strictly message 
passing semantics, I believe that it uses shared memory in the low level 
implementation.

# Steve