Making a persistent HTTP connection

[Alan Kennedy]

[David Rasmussen]
>> I use urllib2 to do some simple HTTP communication with a web server.
>> In one "session", I do maybe 10-15 requests. It seems that urllib2
>> opens op a connection every time I do a request. Can I somehow make it
>> use _one_ persistent connection where I can do multiple GET->"receive
>> data" passes before the connection is closed?

[Diez B. Roggisch]
> Are you sure HTTP supports that?

Yes, HTTP 1.1 definitely supports multiple requests on the same
connection.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8.1

Some HTTP 1.0 clients supported persistent connections through the use
of the non-standard "keep-alive" header.

> And even if it works - what is the problem with connections being created?

The URL above describes the benefits of persistent connections. The
primary problem of the old style of one-request-per-connection is the
creation of more sockets than are necessary.

To the OP: neither urllib nor urllib2 implements persistent
connections, but httplib does. See the httplib documentation page for
an example.

http://www.python.org/doc/2.4.2/lib/httplib-examples.html

However, even httplib is "synchronous", in that it cannot pipeline
requests: the response to the first request must be competely read
before a second request can be issued.

HTH,

--
alan kennedy
------------------------------------------------------
email alan:              http://xhaus.com/contact/alan