Please help with Threading

[Carlos Nepomuceno]

----------------------------------------
> Date: Sun, 19 May 2013 13:10:36 +1000
> From: cs at zip.com.au
> To: carlosnepomuceno at outlook.com
> CC: python-list at python.org
> Subject: Re: Please help with Threading
>
> On 19May2013 03:02, Carlos Nepomuceno <carlosnepomuceno at outlook.com> wrote:
> | Just been told that GIL doesn't make things slower, but as I
> | didn't know that such a thing even existed I went out looking for
> | more info and found that document:
> | http://www.dabeaz.com/python/UnderstandingGIL.pdf
> |
> | Is it current? I didn't know Python threads aren't preemptive.
> | Seems to be something really old considering the state of the art
> | on parallel execution on multi-cores.
> | What's the catch on making Python threads preemptive? Are there any ongoing projects to make that?
>
> Depends what you mean by preemptive. If you have multiple CPU bound
> pure Python threads they will all get CPU time without any of them
> explicitly yeilding control. But thread switching happens between
> python instructions, mediated by the interpreter.

I meant operating system preemptive. I've just checked and Python does not start Windows threads.

> The standard answers for using multiple cores is to either run
> multiple processes (either explicitly spawning other executables,
> or spawning child python processes using the multiprocessing module),
> or to use (as suggested) libraries that can do the compute intensive
> bits themselves, releasing the while doing so so that the Python
> interpreter can run other bits of your python code.

I've just discovered the multiprocessing module[1] and will make some tests with it later. Are there any other modules for that purpose?

I've found the following articles about Python threads. Any suggestions?

http://www.ibm.com/developerworks/aix/library/au-threadingpython/
http://pymotw.com/2/threading/index.html
http://www.laurentluce.com/posts/python-threads-synchronization-locks-rlocks-semaphores-conditions-events-and-queues/


[1] http://docs.python.org/2/library/multiprocessing.html


> Plenty of OS system calls (and calls to other libraries from the
> interpreter) release the GIL during the call. Other python threads
> can run during that window.
>
> And there are other Python implementations other than CPython.
>
> Cheers,
> --
> Cameron Simpson <cs at zip.com.au>
>
> Processes are like potatoes. - NCR device driver manual