HTTP application streams a response to the client gradually - how deal with it?

[Alan Kennedy]

Harald Massa wrote:

> I want to connect to an HTTP application, that streams its response to 
> the client gradually.

Hmmm, this is actually done by HTTP, at a "software timescale", i.e.
when programs are talking to programs. For example, servers send
responses in "chunks", when they don't know how long the final entity is
going to be. The client software assembles the chunks, and presents the
finished whole to the user.

But there is a general assumption in HTTP that the process generating
the output will not delay unnecessarily, and the transaction will be
completed without unnecessary delay. 

> (it is a "web chat")

And if you tried to make the same thing happen at "human timescales", I
think you'd find yourself swimming against the current, i.e. find it
very difficult to do.

> From reading documentes and google, I learned that urllib will not fit.
> Also httplib --- I only see methods that provide me with whole HTTP-
> responses.

That's because HTTP has always been, and probably always will be, a
request/response protocol. One HTTP request (which may transfer several
resources) results in exactly one HTTP response (which may contain
multiple resources).

I think that HTTP is fundamentally mismatched to a chat application,
unless every "sentence" of the chat is POSTed to a central server, and
the clients regularly refresh the content from the central server. It is
not possible for the server to push content to the clients with HTTP.
Clients must initiate the conversation, and must do so regularly if they
want regular updates (this can be automated with meta-refresh tags in
HTML).

However, HTTP is just a protocol layered over sockets. 

I'm sure they're are several other protocols that more closely, or
probably precisely, match your requirements. For example, twisted seems
to have plenty of functionality in this regard:-

http://www.twistedmatrix.com/products/words
http://www.twistedmatrix.com/documents/examples/

If you disagree that HTTP is not suitable for a chat application, please
feel free to say why.

> Is there any way to deal with "streamed http" without going to socket-
> programming?

If you're going to do some socket programming, then the first thing
you'll probably end up doing is giving up on the idea of using HTTP,
which would make your life a *whole* lot simpler.

If your issue is with the use of port 80 (or whatever HTTP port is in
use), often being the only port that goes through firewalls, then that's
a different issue from trying to use HTTP for what it wasn't designed
for.

Hopefully someone will follow up this message with a list of protocols
that are suitable for chat apps. Anyone?

regards,

-- 
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan:              http://xhaus.com/mailto/alan