urlgrabber cookie handling?

[John J. Lee]

Devraj <devraj at gmail.com> writes:

> Hi everyone,
>
> I have been battling to make my code work with a HTTPS proxy, current
> my code uses urllib2 to to most things and works well, except that
> urllib2 doesn't handle HTTPS proxies.
>
> Urlgrabber (http://linux.duke.edu/projects/urlgrabber/help/
> urlgrabber.grabber.html) looks very promising except that I can find a
> way to handle cookies in urlgrabber. Is there a way urlgrabber can use
> a HTTPCookieProcess or cookielib.CookieJar object to handle cookies?

I don't see a nice way.  But then I don't see any HTTPS proxy support
in urlgrabber... (I looked at version 3.1.0).

There is a recipe or two on ASPN showing how to support HTTPS proxies
with urllib2, which gives an idea how to do it, though the code is a
bit rough (I'd post some code myself, but I did it for work).


John