[Web-SIG] question about connection pool, task queue in WSGI

[Roberto De Ioris]

> Hi list,
>
> I am running a site with django + uwsgi, I have few questions about how
> WSGI works.
>
> 1. Is db connection open/close handled by Django? If it's open/closed per
> request, can we make a connection pool in wsgi level, then multiple django
> views can share it?
>
> 2. As a general design consideration, can we execute some task *after* the
> response has returned to client? I have some heavy data processing need to
> be done after return HttpResponse() in django, the standard way to do this
> seems like Celery or other task queue with a broker. It's just too
> heavyweight. Is it possible to do some simple background task in WSGI
> directly?

You can abuse one of the feature you already found in uWSGI.

The simplest approach would be using the Spooler (check uWSGI docs).

It is a simplified celery, where the queue is a simple 'spool directory'
(like a printing system).

A non-uWSGI related trick, would be having a thread pool (one for each
worker) in which you enqueue tasks from the request handler:

http://projects.unbit.it/uwsgi/wiki/Example#threadqueue

There are other solutions to your problem, but all are not relevant to
WSGI, so you may want to move to discussion to the uWSGI list directly.

-- 
Roberto De Ioris
http://unbit.it