Please help with Threading

[Fábio Santos]

My use case was a tight loop processing an image pixel by pixel, or
crunching a CSV file. If it only uses local variables (and probably hold a
lock before releasing the GIL) it should be safe, no?

My idea is that it's a little bad to have to write C or use multiprocessing
just to do simultaneous calculations. I think an application using a
reactor loop such as twisted would actually benefit from this. Sure, it
will be slower than a C implementation of the same loop, but isn't fast
prototyping a very important feature of the Python language?
On 20 May 2013 08:45, "Cameron Simpson" <cs at zip.com.au> wrote:

> On 20May2013 07:25, Fábio Santos <fabiosantosart at gmail.com> wrote:
> | On 18 May 2013 20:33, "Dennis Lee Bieber" <wlfraed at ix.netcom.com> wrote:
> | >         Python threads work fine if the threads either rely on
> intelligent
> | > DLLs for number crunching (instead of doing nested Python loops to
> | > process a numeric array you pass it to something like NumPy which
> | > releases the GIL while crunching a copy of the array) or they do lots
> of
> | > I/O and have to wait for I/O devices (while one thread is waiting for
> | > the write/read operation to complete, another thread can do some number
> | > crunching).
> |
> | Has nobody thought of a context manager to allow a part of your code to
> | free up the GIL? I think the GIL is not inherently bad, but if it poses a
> | problem at times, there should be a way to get it out of your... Way.
>
> The GIL makes individual python operations thread safe by never
> running two at once. This makes the implementation of the operations
> simpler, faster and safer. It is probably totally infeasible to
> write meaningful python code inside your suggested context
> manager that didn't rely on the GIL; if the GIL were not held the
> code would be unsafe.
>
> It is easy for a C extension to release the GIL, and then to do
> meaningful work until it needs to return to python land. Most C
> extensions will do that around non-trivial sections, and anything
> that may stall in the OS.
>
> So your use case for the context manager doesn't fit well.
> --
> Cameron Simpson <cs at zip.com.au>
>
> Gentle suggestions being those which are written on rocks of less than
> 5lbs.
>         - Tracy Nelson in comp.lang.c
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20130520/c2682f10/attachment.html>