urllib2 slow for multiple requests

[cgoldberg]

> Bascally it just grabs a page xy
> times and tells me how long it took.

you aren't doing a read(), so technically you are just connecting to
the web server and sending the request but never reading the content
back from the socket.  So your timing wouldn't be accurate.

try this instead:
response = urllib2.urlopen(req).read()

But that is not the problem you are describing...


> when I increase the number of repetitions, it is
> slowing down considerably (1 is like 3 ms, 100 takes 6 seconds).
> Maybe it is a known issue in urllib2

I ran your code and can not reproduce that behavior.  No matter how
many repetitions, I still get a similar response time per transaction.

any more details or code samples you can provide?

-Corey Goldberg