download x bytes at a time over network

[Saurabh]

I want to download content from the net - in chunks of x bytes or characters
at a time - so that it doesnt pull the entire content in one shot.

import urllib2
url = "http://python.org/"
handler = urllib2.urlopen(url)

data = handler.read(100)
print """Content :\n%s \n%s \n%s""" % ('=' * 100, data, '=' * 100)

# Disconnect the internet

data = handler.read(100) # I want it to throw an exception because it cant
fetch from the net
print """Content :\n%s \n%s \n%s""" % ('=' * 100, data, '=' * 100) # But
still works !

Apparently, handler = urllib2.urlopen(url) takes the entire data in buffer
and handler.read(100) is just reading from the buffer ?

Is it possible to get content from the net in chunks ? Or do I need to use
HTTPClient/sockets directly ?

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20090316/5f76b0d7/attachment.html>