[Web-SIG] Thread-management middleware components?

[Ian Bicking]

Jim Fulton wrote:
>> Right now all threading and generally concurrency is handled by the 
>> server.  Since it *has* to be handled by the server,
> 
> 
> Why does it have to be handled by the server?

Because most WSGI apps are blocking, so unless you want the server to be 
non-concurrent it has to handle this.  Of course you design a 
non-concurrent WSGI server that *had* to be used with some threading 
middleware.  WSGI doesn't seem like a good fit for that, though.

>  > I'm not sure what
> 
>> the advantage would be to duplicating that functionality? 
> 
> 
> The advantage is that it gives people deploying an application
> more control.  We've recently switched to using WSGI for
> HTTP in Zope.  Our default "out of the box" server of choice
> is Twisted, however, the current thread-management strategy used by
> Twisted's WSGI server doesn't meet out needs.  I could try to get
> Twisted to change it's stragegy, and I probably will, but it would
> be more flexible to be able to plug something in.

I think in this particular case -- barring direct changes to Twisted -- 
it would make more sense to build on Twisted's non-WSGI asyncronous 
application support, and build a threadpool that calls WSGI from there.

>  > Well,
>> strictly speaking you could have a server with wsgi.threaded and 
>> wsgi.multiprocess both being false, and the server presumably being 
>> asynchronous, but I think that's challenging to fit into the WSGI spec 
>> -- there was some discussion some time ago that dwindled off, and I 
>> don't think there was ever any resolution on handling asynchronous 
>> servers/apps in WSGI.
> 
> 
> We have long experience with combining an asynchronous network server
> with a threaded application server. Asynchronous network servers can
> handle I/O with lots of network clients very efficiently, but only
> if an application doesn't block.  Real applications often take
> significant time to compute results.  A thread-management facility that
> bridges asychronous servers with threaded application can work very well.
> 
> It's possible that my need is specific to using asynchronous servers,
> but I consider working well with asynchronous servers to be a pretty
> important requirement.

I think the server has to be synchronous by the time it calls a WSGI 
app.  There's nothing saying that the WSGI support in Twisted is the 
WSGI support you have to use.

My impression is that it is hard to standardize anything async-related 
because they use slightly different conventions on how to do async 
(e.g., Deferred vs. ad hoc callbacks).  So... whatever standardization 
there is to be done there is probably below WSGI.

>> I don't see a need for a lot of interchangeable thread pools, a 
>> handful at most should do.
> 
> 
> I'm not sure what you mean by this.
> 
> On the one hand, I'd like to be free to choose my own thread-management
> stragegy.  On the other hand, if there are multiple asynchronous servers,
> I don't see why they should each have to maintain their own 
> thread-management
> subsystems if one can be shared among the different servers.

Sure, but if there's only, say, 4 viable strategies and 3 serious async 
servers (are there even that many of either?) then it's easier just to 
figure out how to plug each strategy in on a case-by-case basis, and 
discuss the concrete issues with the server developers.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org