Async IO Server with Blocking DB

[Damjan Georgievski]

> We are thinking about building a webservice server and considering
> python event-driven servers i.e. Gevent/Tornado/ Twisted or some
> combination thereof etc.
> 
> We are having doubts about the db io part. Even with connection
> pooling and cache, there is a strong chance that server will block on
> db. Blocking for even few ms is bad.

Libraries that support the event loop of your framework, be it Gevent,
Tornado or Twisted, should yield to other work when the database
connection (its socket) blocks.

Now, database drivers that use C libraries are ussually a problem,
unless designed to cooperate with event loops. One of those is psycopg2
which at least cooperates with Gevent. CouchDBkit (using restkit ie
http) also cooperates with Gevent fine.

MySQLdb does not cooperate with Gevent as far as I know, so one
sollution would be to run queries through a Thread Pool (built-in in
Gevent 1.0dev). Another is to use the pure python mysqlconn
(mysql-connector-repackaged on pypi) with gevents monkey patching.

Sqlite should be probably run in a ThreadPool too, since it depends on
disk I/O which is not async in Gevent.


You can readup on Gevent integration of Couchdbkit and Psycopg2 in their
documentation.


> can someone suggest some solutions or is async-io is not at the prime-
> time yet.


-- 
damjan