[Python-ideas] Python and Concurrency

[Josiah Carlson]

Talin <talin at acm.org> wrote:
> I would like to direct your attention to a presentation that was given 
> by Mr. Herb Sutter on the subject of concurrency support in programming 
> languages. It's a long video, but I would urge you to watch at least the 
> first five minutes of it:

I'll watch it later today, but you seem to have done a great job of
explaining what it is saying.

[snip]
> The gist of the introduction is this (my words, not his): Starting from 
> now, and for the rest of your career as a programmer, you better start 
> programming for massively concurrent systems or you are doomed to 
> irrelevance.

Given an inexpensive fork() with certain features (the optional ability
to not copy threads, certain file handles, be able to quickly pass data
between the parent and child processes, etc.), task-level concurrency is
not quite as hard as it is right now. In the same way that one can use a
threadpool to handle queued tasks, one could use whole processes to do
the same thing, which gets us concurrency in Python.


Of course we run into the issue where processor scheduling needs to get
better to handle all of the processes, but that's going to be a
requirement for these multi-core processors anyways.  Windows doesn't
have a .fork() (cygwin emulates it by using a shared mmap to copy all
program state).  Sometimes transferring objects between processes is
difficult (file handles end up being relatively easy on *nix AND Windows),
but there.  Etc.


 - Josiah