HTTP application streams a response to the client gradually - how deal with it?

[Harald Massa]

Alan,

> I.E. You want to be able to read the user input channel 
> at the same time as retrieving updates from the server. And because 
> urllib and  httplib encapsulate the entire request/response inside 
> one method call, your client is blocked while the server call is 
> executing, and thus very unresponsive?

Not exactly the problem ... as you say, that could be easily solved with 
threading.

The exact problem is:

the data has to be processed one chunk at a time. Without the response 
being complete.

Example: The chat server sends:

Bart says: nice to see you all here
Miriam says: yes, I also enjoy the talk
Lisa says: cool evening, yeah
...

the response channel stays open. All the time. When I wait for it to 
finish, so that I can start processing the Response-Objekt - no good.

And while the responses get in from the server, I also have to do 
additional POST requests

POST user=Icke says="i also think it's a nice evening"

> There is a simple solution to that problem: use multiple threads. One
> thread reads input from the user, another thread does nothing but
> contact the server periodically for updates.
Yeah, that would be perfect. But: it seems that the server does not 
finish his response. So until httplib.read() has a timeout ...

> Given what you've said, I'd say it's probably overkill to go down to 
> the socket level. 
I surely hope so, Alan. Posting this question is based on my thinking, 
that I can't beat Timbot or Guidocode by any means - and better use it.

> functionality you need, and (mostly) bug free, having been used by
> thousands of people over the years. Rolling your own HTTP protocol
> would be an enormous amount of work, due to the wide range of weird and
> wonderful ways in which HTTP is used and abused.

Yeah ... and I think I have just stumbled into a very abusive one.

Thanks for all your feedback... maybe there is an easier way?

Harald