urllib.urlopen() with pages that requires cookies.

[Fredrik Lundh]

Øyvind Østlund wrote:

> I am trying to visit a limited amount of web pages that requires cookies. I
> will get redirected if my application does not handle them. I am using
> urllib.urlopen() to visit the pages right now. And I need a push in the
> right direction to find out how to deal with pages that requires cookies.
> Anyone have any idea how to go about this?

use urllib2 and cookielib.  here's an outline:

    import urllib2, cookielib

    # set things up
    jar = cookielib.CookieJar()
    handler = urllib2.HTTPCookieProcessor(jar)
    opener = urllib2.build_opener(handler)
    urllib2.install_opener(opener)

    data = urllib2.urlopen(someurl).read()

    # ...

    data = urllib2.urlopen(someotherurl).read()

    # ...

    # dump cookie jar contents (for debugging)
    for cookie in jar:
        print cookie

</F>