[Tutor] Is anybody out there who could help me with URL Authentication?

[Monika Jisswel]

Here is some code, it will help you manage cookies, & stay logged in to the
website for further queries.

import os, sys, time, urllib, urllib2, cookielib, re
>
> cj=cookielib.LWPCookieJar()
> headers={'User-Agent' : user_agent}
> opener=urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
> urllib2.install_opener(opener)
>
> get_login=urllib2.urlopen('https://www.website.com/login_page.html')
>
> recieved_login_page = get_login.readlines()
>
> for line in recieved_login_page:
>     #you've got to parse each line in the page
>     #and extract information from it & assign it to variables
>
> #those variables you just extracted you've got to put them in this
> dictionary
> values={'username':'xxxxxxxxx', 'pass':'xxxxxxxxx', 'submit' : 'submit',
> 'anothervar':'xxxx'}
> data=urllib.urlencode(values)
>
> req=urllib2.Request('https://www.website.com/form_action.php', data,
> headers)  #I think "https://investor.firstrade.com/firstrade/login.do"
> response=urllib2.urlopen(req)
>

ofcourse this is just example code and you should replace some variables
with your own, but it should work for your case if you know where the form
submits its data & what data it submits (you put this data in the values
dict  & then encode it & use it to authenticate and the opener
(urllib2.install_opener(opener)) should keep you logged in to the page
because it keeps track of cookies for you.


2008/8/1 Federo <federo at email.si>

> Hi ..
>
> I have to admit that Python is really surprising me. It was lucky day a few
> weeks ago I firts time start using Python. Lot's of things realy can be
> done
> with short learning curve. Your user guieds was best place to start!
>
> Below is problem I am unable to solve. I would appreciate your advice or
> even better code sample. The problem is URL authorisation. I try to
> approaches
> but no luck so far. Two problems:
>
> 1.) Being able to logon using Python code
> 2.) Being able to send data to a form and get back server reply
>
> ad 1.)
> The Two pages I try to open:
>
> Login:
> https://investor.firstrade.com/firstrade/login.do
> Main page after login:
> https://investor.firstrade.com/firstrade/mainmenu.do
>
> I have tried bellow code (with currect user name and password of course):
> >>> import urllib2
> >>> password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
> >>> protocol = 'https://'
> >>> username = 'user'
> >>> password = 'pass'
> >>> top_level_url = "investor.firstrade.com/firstrade/login.do"
> >>> password_mgr.add_password(None, top_level_url, username, password)
> >>> handler = urllib2.HTTPBasicAuthHandler(password_mgr)
> >>> opener = urllib2.build_opener(handler)
> >>> a_url = "https://investor.firstrade.com/firstrade/login.do"
> >>> f = opener.open(a_url)
> >>> a = f.read()
> >>> print (a)
>
> but it replied login page after login. It looks like they are using cookie.
> Bellow line can be find in header
> Set-Cookie: JSESSIONID=351CE048D9FEAD01E21604129D38725C; Path=/; Secure
>
> On the web I found user guide explaining coookie login. However I do not
> understand how to combine now password and cookie login if this will solve
> the
> problem at first place...
>
> http://www.voidspace.org.uk/python/articles/cookielib.shtml#fetching-webpages
>
> CODE SAMPLE to be able to solve the problem very appreciated..
>
> There is by pass solution using Web Recorder Macro. However it will be much
> better to BE ABLE TO handle entire process with Python..
>
> ad 2.) Filling data to form / getting beck server values. I need server
> reply
> from the same web side we are discussing in point 1.
>
> Here is Code sample:
> import urllib
> import urllib2
>
>    url = 'https://investor.firstrade.com/firstrade/mainmenu.do'
> user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
>    values = {'symbol' : 'xsnx'}
>
>    data = urllib.urlencode(values)
>    req = urllib2.Request(url, data)
>    response = urllib2.urlopen(req)
>    the_page = response.read()
>
> a.) I do not understand where can I find currect fields names for values
> in the above code sample (in our case "Symbol" is written before Text box
> (look
> attached print screen. HTML code of main side is attached file
> Firstrade_MainSideHTML.rtf)
> where you can write stock ticker and than press a button. Server replied
> real
> time stock data which I would like to read)?
> b.) Probably code will be more complex as sample code above as code has to
> take
> care also for proper authorisation..
>
> Sorry for taking your time. Your help would be very useful as I stacked
> with
> this. Being able to handle URL requests is very important now days...
>
> Cheers, Fedo
>
> ____________________
> http://www.email.si/
>
> ____________________
> http://www.email.si/
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20080801/6a2d4ef2/attachment-0001.htm>