[concurrency] Talk worth watching

[Jeremy McMillan]

I've been trying to figure this stuff out for a couple of years, and this
is as far as I've got in my musings.

Python has lots of functional programming features, but even in functional
languages, "side effects" impacting state outside the function's internal
scope require "heroic" efforts (like locking or message
passing/serialization) to avoid breaking parallel execution or
concurrency/responsiveness.

The declarative programming style makes it trivial to graph which code
cares about which objects in memory. That forces the programmer to solve
the problems a priori, which as it turns out, are the real issue with
effectively getting rid of the GIL in Python, thus ending this debate.

http://dabeaz.blogspot.com/2011/08/inside-look-at-gil-removal-patch-of.html

What we need is smarter memory management, but I think it needs to be
somewhere between the Spartan approach of functional languages and the
relatively cushy GC'ed object-oriented languages.

I wonder if there isn't a perspective available from the AST or something
that would allow cherry-picking sections of logic which can be decomposed
more or less like what you'd end up with from strict functional code, and
possibly executing them in another context (interpreter context, thread,
greenlet?) which would allow releasing the GIL. Or possibly the opposite,
and cherry pick sections of code which must acquire the GIL?

The end result is that when you program in Python, but you're careful to
code in a style that allows embarrassingly parallel execution, you get some
reprieve from the GIL. There has to be some way to guarantee that no code
outside your intended parallel context can ever refer to anything inside,
and then side-effects when you can't avoid them, will block waiting for the
GIL. One practical example would be that using map() and reduce() in Python
would automatically parallelize execution across multiple CPUs.

On Sat, Feb 18, 2012 at 11:02 AM, Andrey Popp <8mayday at gmail.com> wrote:

> On 18.02.2012, at 17:19, Mike Meyer <mwm at mired.org> wrote:
>
> > If you haven't seen it, this is probably worth a gander:
> > http://yow.eventer.com/events/1004/talks/1055
> >
> > It's a research scientist discussing his plans for concurrency in
> > Haskell. I think his goals - and ideas - are pretty solid.
>
> These thing are already implemented in Haskell and more or less
> practically usable.
>
> > However,
> > since he's doing research, he can afford to use an obscure language
> > that makes his life easier. Getting this stuff working in a practical
> > language like Python will be a bit more difficult.
> >
> >    <mike
> > --
> > Mike Meyer <mwm at mired.org>        http://www.mired.org/
> > Independent Software developer/SCM consultant, email for more
> information.
> >
> > O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
> > _______________________________________________
> > concurrency-sig mailing list
> > concurrency-sig at python.org
> > http://mail.python.org/mailman/listinfo/concurrency-sig
> _______________________________________________
> concurrency-sig mailing list
> concurrency-sig at python.org
> http://mail.python.org/mailman/listinfo/concurrency-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/concurrency-sig/attachments/20120218/1137af9f/attachment.html>