Cookies and CookieJar

[7stud]

On May 16, 9:21 am, Larry Bates <larry.ba... at websafe.com`> wrote:
> I'm struggling with a project using mechanize and cookies to screen scape a
> website.  The site requires a client created cookie for authentication.  Below
> is the code I'm attempting to use with the traceback I'm getting:
>
>  >>> import Cookie
>  >>> c=Cookie.SimpleCookie()
>  >>> c["Manageopen"]="cards"
>  >>> c['Manageopen']['expires'] = 0
>  >>> c['Manageopen']['path'] = "/"
>  >>> c['Manageopen']['domain'] = ".domain.com"
>  >>> c['Manageopen']['secure'] = ""
>  >>> c.output()
> 'Set-Cookie: Manageopen=cards; Domain=.domain.com; expires=Fri, 16-May-2008
> 14:06:00 GMT; Path=/'
>  >>> import cookielib
>  >>> cj=cookielib.CookieJar()
>  >>> cj.set_cookie(c)
> Traceback (most recent call last):
>    File "<interactive input>", line 1, in <module>
>    File "C:\Python25\lib\cookielib.py", line 1617, in set_cookie
>      if cookie.domain not in c: c[cookie.domain] = {}
> AttributeError: 'SimpleCookie' object has no attribute 'domain'
>  >>>
>
> I also tried:
>
>  >>> cj.set_cookie(c["Manageopen"])
> Traceback (most recent call last):
>    File "<interactive input>", line 1, in <module>
>    File "C:\Python25\lib\cookielib.py", line 1617, in set_cookie
>      if cookie.domain not in c: c[cookie.domain] = {}
> AttributeError: 'Morsel' object has no attribute 'domain'
>
> I've looked at everything I can find via Google and nothing seems to help.
>
> Thanks in advance.
>
> Regards,
> Larry

Hi,

In the python docs, set_cookie() is defined like this:

---
set_cookie(cookie)
Set a Cookie, without checking with policy to see whether or not it
should be set.
---

The Cookie mentioned there is a an object of the Cookie class which is
defined in the cookielib module.  On the other hand,
Cookie.SimpleCookie() creates an object of a class called SimpleCookie
(and SimpleCookie inherits from a class called BaseCookie).
Therefore, the object returned by Cookie.SimpleCookie() is not a
Cookie--it's a SimpleCookie.  Just because the module that contains
the SimpleCookie class is called Cookie does not mean that
SimpleCookie inherits from the Cookie class or that SimpleCookie has
any connection to the Cookie class located in the cookielib module.
If that's too confusing, here is the way the python modules are set
up:

Cookie module:
-------------
BaseCookie class
    |
    V
SimpleCookie class




cookielib module
----------------
Cookie class

CookieJar class
---set_cookie(Cookie)


What you did in your code was you supplied set_cookie() with a
SimpleCookie object, and set_cookie() expects a Cookie object.
SimpleCookie objects are a mapping and you access the data using the
notation:

>  >>> c['Manageopen']['domain'] = ".domain.com"

But if you open up cookilib.py and look at the line in the error
message:

> File "C:\Python25\lib\cookielib.py", line 1617, in set_cookie

you will see that the code uses the notation:

>      if cookie.domain ...

That's because the Cookie class defines simple objects that have
attributes that are accessed normally, i.e. with dot notation.


> The site requires a client created cookie for authentication.

A "cookie" is just one of the headers in a request.  Instead of using
a url string to retrieve a web page with urllib2.urlopen(), you can
use a Request object.  And Request objects allow you to set headers,
which means you can set a cookie header.

But the site will probably check every subsequent request for that
cookie header as well, so that means you will need to add the header
to every subsequent request you make.  That is where CookieJar can
help you.  It can store cookies in a file and then automatically add
them to every request for you.  It's a bit complicated though.  See
here:

http://www.voidspace.org.uk/python/articles/cookielib.shtml