How to keep cookies when making http requests (Python 2.7)

Luca Cerone luca.cerone at gmail.com
Tue Aug 27 06:17:54 EDT 2013


> 
> > Let me make an additional remark however: you should
> > not expect to get complete details in a list like this - but only
> > hints towards a solution for your problem (i.e. 
> > there remains some work for you).
> > Thus, I expect you to read the "cookielib/cookiejar" documentation
> > (part of Python's standard documentation) in order to understand
> > my example code - before I would be ready to provide further details.

Ok so after reading the documentation for urllib2 and cookielib I came up with the following code:

#START
from urllib2 import urlopen , Request
from cookielib import CookieJar
import re
regex = re.compile(r'<span class=\'x\'>\{(.*)\}<span class=\'x\'>')

base_url = "http://quiz.gambitresearch.com"
job_url  = base_url + "/job/"

cookies = CookieJar()
r = Request(base_url) #prepare the request object
cookies.add_cookie_header(r) #allow to have cookies
R = urlopen(r) #read the url
cookies.extract_cookies(R,r) #take the cookies from the response R and adds #them to the request object 

#build the new url
t = R.read()
v = str(eval(regex.findall(t)[0]))
job_url = job_url + v


# Here I create a new request to the url containing the email address
r2 = Request(job_url)
cookies.add_cookie_header(r2) #I prepare the request for cookies adding the cookies that I extracted before.

#perform the request and print the page
R2 = urlopen(r2)
t2 = R2.read()
print job_url
print t2
#END

This still doesn't work, but I really can't understand why.
As far as I have understood first I have to instantiate a Request object
and allow it to receive and set cookies (I do this with r = Request() and cookies.add_cookie_header(r))
Next I perform the request (urlopen),  save the cookies in the CookieJar (cookies.extract_cookies(R,r)).

I evaluate the new address and I create a new Request for it (r2 = Request)
I add the cookies stored in the cookiejar in my new request (cookies.add_cookie_header(r2))
Then I perform the request (R2 = urlopen(r2)) and read the page (t2 = R2.read())

What am I doing wrong? Do I misunderstand something in the process?

Thanks again in advance for the help,
Cheers,
Luca



More information about the Python-list mailing list