waiting for html to load: a followup

[John J. Lee]

Josh <joshl at commenspace.org> writes:
[...]
> Anyway, I decided to forget IE and I am now trying to use urllib2 to
> open up the page, read it, etc.  My problem is the page has a built-in
> refresh and I don't know how to have python re-read the page until
> it's ready to hand over the links.

ClientCookie does that (HTTPRefreshProcessor and HTTPEquivProcessor in
particular).

http://wwwsearch.sf.net/ClientCookie


I recommend using the alpha release.  The interface will change a
little soon, but you almost certainly won't notice.


> An example of the page is:
> http://edcw2ks23.cr.usgs.gov/Website/zipship/waiting.jsp?areaList=49.0,47.0,-122.0,-124.08&prodList=NED,
> 
> I believe I need to read the header, grab the cookie session id, and
> add it back to the header.  I can do all thus, but I'm stuck on

It'll do the cookies too :-)

[...]
> probably very simple syntax to re-read the page rather than open a new
> connection, if that makes sense (I'm new to http as well as python).

You don't need to ensure it's the same connection.  In fact, you can't
easily do that with urllib2 (or ClientCookie) as it is currently.

HTH


John