[Python-ideas] Cofunctions: It's alive! Its alive!

[ghazel at gmail.com]

On Sat, Aug 7, 2010 at 3:05 AM, Carl M. Johnson
<cmjohnson.mailinglist at gmail.com> wrote:
>
> On Fri, Aug 6, 2010 at 9:12 PM, Greg Ewing wrote:
>
> > I've been doing some more hacking, and I now have a
> > working implementation of cofunctions, which I'll
> > upload somewhere soon.
> >
> > I have also translated my yield-from examples to
> > use cofunctions. In the course of doing this, the
> > additional restrictions that cofunctions impose have
> > already proved their worth -- I forgot a cocall, and
> > it clearly told me so and pointed out exactly where
> > it had to go!
>
> This is good to hear. Without being too critical, I feel like saying
> that so far I've been following the cofunctions threads and waiting
> for compelling use cases. So, I'll be happy to see what you have
> there. It seems like the main use case that comes to mind for
> cofunctions is, essentially, quick and dirty cooperative multitasking.
> Of course, as we all know on the OS side of things, cooperative
> multitasking has been more or less phased out (I don't know about the
> embeded space. Probably it's hanging on there for real time purposes.)
> in favor of preemptive multitasking. But Python already has preemptive
> multitasking: it's called threads. Or, if one prefers, there is
> multiprocessing. Of course, those are relatively heavy-weight, but
> then again, so is adding new keywords and syntax. So, other use cases
> would be appreciated.

I am excited about the cofunctions PEP and features, since it will
greatly improve the ability for monocle to have cooperative async
tasks. An extension in the yield-from PEP allows us to return normal
values without confusing semantics, cocall and codef allow us to avoid
several types of errors, and in general the idea of a cofunction is
promoted to a first-class tool, instead of an overloaded trick on top
of generators.

This need for these monocle chains is a much larger topic, but threads
and processes introduce locking and state sharing complications which
are much more involved than simply pausing a function until data is
received. Not to mention the GIL. Cooperative multitasking in the OS
was set aside, but async IO is alive and well. Allowing programmers to
use async IO with a linear blocking look-alike syntax seems to be an
important compromise which makes it easier to write efficient
concurrent networking code.

-Greg