[stdlib-sig] futures - a new package for asynchronous execution

[Jesse Noller]

On Fri, Nov 6, 2009 at 8:12 PM, Brett Cannon <brett at python.org> wrote:
> [I am going to be lazy and mass reply to people with a top-post; you
> can burn an effigy of me later]
>
> In response to Guido, yes, I sent Brian here with my PEP editor hat
> on. Unfortunately the hat was on rather firmly and I totally forgot to
> check to see how old the code is. Yet another reason I need to get the
> Hg conversion done so I can start writing a "Adding to the Stdlib"
> PEP.

Want to start the skeleton and we can fill in the blanks as needed? :)
I vote for "The Getting Your Pony into Python" PEP.

> To Antoine's Twisted comment, I don't see a direct comparison. From my
> understanding Twisted's Deferred objects are ways to have callbacks
> executed once an async event occurs, not to help execute code
> concurrently. So I don't see enough similarity to discount the idea
> Brian is pushing forward.

It's also significantly less code than many existing solutions; it's
really light when compared to those as well.

> As for Benjamin's one year reminder, since I don't see this happening
> in time for Python 3.2a1 it isn't a major worry right now. That means
> this won't land until Python 3.3, which gives everyone time until that
> alpha which would probably be June 2011. So that would give Brian (and
> Jesse if he gets involved like it sounds he will) time to work out the
> API, get public feedback, and get the code checked in. The only way I
> would feel comfortable letting this in past 3.2a1 would be if it
> landed before 3.2a4, Jesse personally shuttled it through, and started
> work in it now and REALLY pushed it. But as he knows from personal
> experience, rushing a module into the stdlib can bite you in the ass.
> =)

I don't want to rush it; I think with tests and more examples and a
run through the python-ideas and -dev spin cycles, it's small enough
(and consumable enough) to get into the 3.2 stream. But just to
stress, I'm really not interested in rushing it, I just don't see it
as something as "big" and potentially as big a disruption as
multiprocessing was.

Again, I consider this more akin to the addition of the same API just
added to the threading or multiprocessing modules. My pony list for
both of those have been built in producer/consumer pools, things like
this, more context manager things, etc. Build on the modules to
implement common constructs.

> But I really do like the idea. With java.util.concurrent and Grand
> Central Dispatch out there, I think it shows some demand for a way to
> easily abstract out concurrency management stuff and leave it up to a
> library.

Making it eas(y)(ier), safer and simple would be nice. There's plenty
of tried and true patterns out there that just make sense and that
plenty of people implement so frequently that they just sort of scream
(to me, I might be in the minority) for inclusion in the relevant
modules.

Who knows, this could be the start of python.concurrent :)

jesse