[Web-SIG] Thread-management middleware components?

[Jim Fulton]

Ian Bicking wrote:
> Jim Fulton wrote:
> 
>> Has anyone written any thread-management middleware components for WSGI?
>> Many web applications need to run application code in separate threads.
>> Often, the number of threads needs to be limited, either by throttling
>> the rate of thread creation, or by dispatching requests to a thread pool.
>> This is a capability that could be provided by a server, however, it 
>> seems
>> that it might be functionality better provided at an intermediate 
>> layer to
>> make it more pluggable.
> 
> 
> Right now all threading and generally concurrency is handled by the 
> server.  Since it *has* to be handled by the server,

Why does it have to be handled by the server?

 > I'm not sure what
> the advantage would be to duplicating that functionality? 

The advantage is that it gives people deploying an application
more control.  We've recently switched to using WSGI for
HTTP in Zope.  Our default "out of the box" server of choice
is Twisted, however, the current thread-management strategy used by
Twisted's WSGI server doesn't meet out needs.  I could try to get
Twisted to change it's stragegy, and I probably will, but it would
be more flexible to be able to plug something in.


 > Well,
> strictly speaking you could have a server with wsgi.threaded and 
> wsgi.multiprocess both being false, and the server presumably being 
> asynchronous, but I think that's challenging to fit into the WSGI spec 
> -- there was some discussion some time ago that dwindled off, and I 
> don't think there was ever any resolution on handling asynchronous 
> servers/apps in WSGI.

We have long experience with combining an asynchronous network server
with a threaded application server. Asynchronous network servers can
handle I/O with lots of network clients very efficiently, but only
if an application doesn't block.  Real applications often take
significant time to compute results.  A thread-management facility that
bridges asychronous servers with threaded application can work very well.

It's possible that my need is specific to using asynchronous servers,
but I consider working well with asynchronous servers to be a pretty
important requirement.

> I don't see a need for a lot of interchangeable thread pools, a handful 
> at most should do.

I'm not sure what you mean by this.

On the one hand, I'd like to be free to choose my own thread-management
stragegy.  On the other hand, if there are multiple asynchronous servers,
I don't see why they should each have to maintain their own thread-management
subsystems if one can be shared among the different servers.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org