[Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

[Krishna Sankar]

Guido,
    The vagueness is deliberate, to  keep the options open until we have 
some form o convergence. Parallelism/concurrency is a vast and important 
domain that I do not want to develop a hasty proposal. But I did want to 
start thinking in terms of concrete proposals, not pontifying, hence the 
"pragmatic" section.
    Happy to hear that you are open to PVM changes. It will not be asked 
unless and until we all are crisp about it.
Cheers
<k/>
   
Guido van Rossum wrote:
> On 9/18/07, Krishna Sankar <ksankar at doubleclix.net> wrote:
>   
>> Folks,
>>    As a follow-up to the py3k discussions started by Bruce and Guido, I
>> pinged Brett and he suggested I submit an exploratory proposal. Would
>> appreciate insights, wisdom, the good, the bad and the ugly.
>>
>> A)    Does it make sense ?
>> B)    Which application sets should we consider in designing the
>> interfaces and implementations
>> C)    In this proposal, parallelism and concurrency are used in an
>> interchangeable fashion. Thoughts ?
>> D)    Please suggest pertinent links, discussions and insights.
>> E)    I have kept the proposal to a minimum to start the discussions and
>> to explore if this is the right thing to do. Collaboratively, as we
>> zero-in on one or two approaches, the idea is to expand it to a crisp
>> and clear PEP. Need to do some more formatting as well.
>>     
>
> I'd say it is a little light on specific proposals. The only section
> that actually proposes anything is this:
>
>   
>> Pragmatic proposal
>> ------------------
>> May I suggest we embed two primitives in Python 3K:
>> A)    A functional style share-nothing set of interfaces (and
>> implementations thereof) - provides  the task parallelism/concurrency
>> capability, "small messages, big computations" as Joe Armstrong calls it[3]
>> B)    A limited shared memory based model for data intensive parallelism
>>
>> Most probably this would be part of stdlib. While Guido is almost right
>> in saying that this is a (std)library problem, it is not fully so. We
>> would need a few primitives from the underlying PVM substrate. Possibly
>> one reason for Guido's position is the lack of clarity as to what needs
>> to be changed and why. IMHO, just saying take GIL off does not solve the
>> problem either.
>>     
>
> Before I can meaningfully comment I think I'd like to hear more about
> what specifically you are thinking of.
>
> I don't mind the necessary changes to the PVM. I do like to see how
> this affects existing C extensions though.
>
>