Event driven server that wastes CPU when threaded doesn't

[Jean-Paul Calderone]

On 29 Oct 2006 13:13:32 -0800, Nick Vatamaniuc <vatamane at gmail.com> wrote:
>Snor wrote:
>> I'm attempting to create a lobby & game server for a multiplayer game,
>> and have hit a problem early on with the server design. I am stuck
>> between using a threaded server, and using an event driven server. I've
>> been told time and time again that I should use an event driven server
>> design (that is, use twisted).
>>
>> There is a lot of interaction between the clients and they would often
>> need to write to the same list of values, which of course becomes a
>> problem with a threaded server - so event driven solves that problem,
>> and I assumed it would solve all my problems. However some requests
>> from clients would require that the server goes on to query a mySQL
>> server (separate machine from the server). As this occurs, there is a
>> small amount of lag while the communication with the mySQL server takes
>> place, and there could be another 100 clients waiting to make a request
>> at this point, meanwhile the server is idling while waiting for a
>> response from the mySQL server - obviously not a good server model.
>>
>> I will want the server to support as many users as is possible on any
>> given machine - and so wasted CPU cycles is something I am trying to
>> avoid.
>>
>> Is the only solution to use a threaded server to let my clients make
>> their requests and receive a response in the fastest possible time?
>Snor,
>
>The simplest solution is to change your system and put the DB on the
>same machine thus greatly reducing the time it takes for each DB query
>to complete (avoid the TCP stack completely).  This way you might not
>have to change your application logic.
>
>If that is not an option, then you are faced with a problem of
>connecting a threaded programming model with an event based model
>(twisted and and such). So your job is to interface the two. In other
>words make the event based model see the threaded DB access as event
>based. And the DB driver to see the event-based system as threaded. So
>in your event dispatcher you could add events like db_request_finished
>then when a connection is requested to the DB, a callback will be
>supplied. The connection will take place in its own thread, then when
>it is finished it will put the db_request_finished and the respective
>callback function on the event queue. I am not sure how to integrate
>that into the Twisted event dispatcher... perhaps this class is the
>answer?:
>http://twistedmatrix.com/documents/current/api/twisted.python.dispatch.EventDispatcher.html

Note, however:

>>> from twisted.python import dispatch
__main__:1: DeprecationWarning: Create your own event dispatching mechanism, twisted.python.dispatch will soon be no more.

Take a look at <http://twistedmatrix.com/documents/current/api/twisted.enterprise.adbapi.html>.

Jean-Paul