urllib and persistence

[Alex Martelli]

Mark Carter wrote:

>> >   http://wwwsearch.sourceforge.net/ClientCookie/ may thus
>> > prove of help.
>> 
>> As will wget, if you want a non-python solution.
> 
> Alas, the site requires logging in, but not in a strict
> account/password way - it wants
> to know your surname, for example. So I don't think that wget will
> help.

Actually wget is able to load cookies from Netscape-format cookies.txt
files, so if you did a first login with Netscape you could then use
wget with the --load-cookies option.  I read on wget's manual, but
haven't tried myself, that from Internet Explorer you can produce a
textfile that wget can use:

"""
You can produce a cookie file Wget can use by using the File menu, Import 
and Export, Export Cookies. This has been tested with Internet Explorer 5; 
it is not guaranteed to work with earlier versions. 
"""

There are a few recipes in the Python Cookbook for dealing with
cookies as saved by both Netscape and IE.


> I must admit that I'm a bit lacking in understanding how urllib2
> and/or ClientCookie
> are supposed to work. Does ClientCookie work with Internet Explorer
> cookies, or not?

MSIECookies SHOULD let you use cookies saved by IE -- I think it's
prominently enough explained among the FAQ's at
http://wwwsearch.sourceforge.net/ClientCookie/ .  Using netscape-
formatted cookies may be preferable even there, though -- better
tested/clearer/more understood (as it's text, it's easier to
understand and "reverse engineer".



Alex