Fastest way to retrieve and write html contents to file

DFS nospam at dfs.com
Mon May 2 21:52:04 EDT 2016


On 5/2/2016 4:42 AM, Peter Otten wrote:
> DFS wrote:
>
>>> Is VB using a local web cache, and Python not?
>>
>> I'm not specifying a local web cache with either (wouldn't know how or
>> where to look).  If you have Windows, you can try it.
>
> I don't have Windows, but if I'm to believe
>
> http://stackoverflow.com/questions/5235464/how-to-make-microsoft-xmlhttprequest-honor-cache-control-directive
>
> the page is indeed cached and you can disable caching with
>
>> Option Explicit
>> Dim xmlHTTP, fso, fOut, startTime, endTime, webpage, webfile,i
>> webpage = "http://econpy.pythonanywhere.com/ex/001.html"
>> webfile  = "D:\econpy001.html"
>> startTime = Timer
>> For i = 1 to 10
>> Set xmlHTTP = CreateObject("MSXML2.serverXMLHTTP")
>> xmlHTTP.Open "GET", webpage
>
>   xmlHTTP.setRequestHeader "Cache-Control", "max-age=0"


Tried that, and from later on that stackoverflow page:

xmlHTTP.setRequestHeader "Cache-Control", "private"

Neither made a difference.  In fact, I saw faster times than ever - as 
low as 0.41 for 10 loops.



More information about the Python-list mailing list