Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Sun, May 1, 2016, at 10:59 PM, DFS wrote:
> startTime = time.clock()        
> for i in range(loops):
> 	r = urllib2.urlopen(webpage)
> 	f = open(webfile,"w")
> 	f.write(r.read())
> 	f.close
> endTime = time.clock()          
> print "Finished urllib2 in %.2g seconds" %(endTime-startTime)

Yeah on my system I get 1.8 out of this, amounting to 0.18s. 

I'm again going back to the point of: its fast enough. When comparing
two small numbers, "twice as slow" is meaningless.

You have an assumption you haven't answered, that downloading a 10 meg
file will be twice as slow as downloading this tiny file. You haven't
proven that at all. 

I suspect you have a constant overhead of X, and in this toy example,
that makes it seem twice as slow. But when downloading a file of size,
you'll have the same constant factor, at which point the difference is
irrelevant. 

If you believe otherwise, demonstrate it.

-- 
Stephen Hansen
  m e @ i x o k a i . i o