multitask http server (single-process multi-connection HTTP server)

[Luke Kenneth Casson Leighton]

On Tue, Jul 13, 2010 at 1:30 AM, Tim Wintle <tim.wintle at teamrubber.com> wrote:
> On Mon, 2010-07-12 at 23:28 +0000, Luke Kenneth Casson Leighton wrote:
>> On Mon, Jul 12, 2010 at 10:13 PM, geremy condra <debatem1 at gmail.com> wrote:
>> > On Mon, Jul 12, 2010 at 4:59 PM, lkcl <luke.leighton at gmail.com> wrote:
>> >> there probably exist perfectly good web frameworks that are capable of
>> >> doing this sort of thing: i feel certain that twisted is one of them.
>> >> however, the original author of rtmplite decided to rip twisted out
>> >> and to use multitask.py and i'm one of those strange people that also
>> >> likes the idea of using 900 lines of awesome elegant code rather than
>> >> tens of thousands of constantly-moving-target.
>
> have you seen nagare:
> http://www.nagare.org/

 i have now! :)

 it uses stackless python, which is proobbably where the nonblocking
aspects come from.  going from there...

 http://stacklessexamples.googlecode.com/svn/trunk/examples/networking/basicWebserver.py

 ah ha!  on the face of it, that does actually look like it achieves
the same sort of thing.

> I've not used it - but from my understanding it might be what you're
> looking for (for the http part at least).

 yes, for the http part: the rest - mmm no.

>> i hate to think how this would be done using any of the standard
>> MixIns.  even if you wrote a special MixIn which did single-instance
>> socket handling, you couldn't use it because the BaseHTTPHandler
>> doesn't "cooperate", it has a while True loop on serving connections
>> until they're closed.
>
> I started working on something like this once (but I still had threads)
> - afraid I can't find the code with it in right now. I think it was
> similar to what you're doing:
>
> at least 2 threads - one accepts requests, the other deals with them.

 ok, that sounds like the problem has moved: requests could still be
received rather than blocked at the TCP level, but they'd still not
actually get processed if the 2nd "dealing with it" thread was in
"serve_forever()" mode.  and because of HTTP Keep-Alives, when
close_connection=1 in the HTTPRequestHandler base class, that would
still be an issue.

 looking at that stackless basic web server example, i believe that
that's actually it: the concept of "tasklets", and that cooperative
scheduling loop:
        while time.time() < t + delay:
            stackless.schedule()

 multitask.py effectively does the same thing, but using "yield",
which is just amazing.

but... not being funny or anything, but basically i'm done already :)
multitaskhttpd works, it doesn't need stackless, i completed a JSONRPC
service last night, i'll add POST of multi-part forms today, and i
have everything that [GNUmed] will need.

i think convincing the gnumed team to get all their users to install
and use stackless python would be a bit of a hard sell.

l.