Outbound HTML Authentication

[Chris Mellon]

On Nov 29, 2007 2:22 PM, Mudcat <mnations at gmail.com> wrote:
> Hi,
>
> I was trying to do a simple web scraping tool, but the network they
> use at work does some type of internal authentication before it lets
> the request out of the network. As a result I'm getting the '401 -
> Authentication Error' from the application.
>
> I know when I use a web browser or other application that it uses the
> information from my Windows AD to validate my user before it accesses
> a website. I'm constantly getting asked to enter in this info before I
> use Firefox, and I assume that IE picks it up automatically.
>
> However I'm not sure how to tell the request that I'm building in my
> python script to either use the info in my AD account or enter in my
> user/pass automatically.
>

You can configure a proxy for urllib2, but your proxy probably uses
NTLM authentication which urllib2 doesn't support. Your best bet is to
use a local proxy which understands NTLM.