Web authentication

[John J. Lee]

"luigipaioro" <luigipaioro at libero.it> writes:

> Good morning to all!

Good morning!  No need to post twice, BTW.


> I'm trying to access on a web page that needs user and password
> authentication. I'm enabled to access there (I mean that I have an
> user name and a password to access via web), but I cannot access using
> an automatic procedure (that is what I need to make a daemon that
> downloads weekly an ASCII file from that site).
> 
> I've tried using urllib:
> 
> import urllib
> 
> conn = urlib.urlopen("http://user:password@www.mysite.com")
> print conn.read()
> 
> But it doesn't work (it asks me again user and password).

That URL should work for "Basic HTTP authentiation" using urllib (I
think -- I always use urllib2, so not certain about urllib).  For some
reason, a quick glance at the code suggests it *won't* work with
urllib2, but it's easy enough to achieve the same result with that
module (see the link below for how).  The page you're accessing may
need some other means of authentication, though.

When you log in manually, does your browser pop up a little, rather
plain-looking, separate window?  Or do you type directly into a form
on the web page itself?  If the former, it's probably Basic auth., and
what you're doing should work (or, unlikely, Digest auth., in which
case I think you need urllib2).  If the latter, you probably need to
submit an HTML form in the web page to log in.

Some examples on auth and proxies with urllib2 (beware: I don't use a
proxy or basic / digest auth. very often, so these are untested
examples: if you use them, *please* comment on them to say whether
they do or do not work as advertised):

http://www.python.org/sf/798244


To fill in HTML forms, you can use urllib2.urlopen(url,
urllib.urlencode(...read the docs <wink>...)), or, if you want Python
to parse the form(s) for you and/or don't want to know the messy
details of HTML forms, you could use

http://wwwsearch.sf.net/ClientForm/

You may also find you need to handle HTTP cookies:

http://wwwsearch.sf.net/ClientCookie/


John