urllib, can't seem to get form post right

Adam W. awasilenko at gmail.com
Thu Sep 24 17:18:25 EDT 2009


I'm trying to scrape some historical data from NOAA's website, but I
can't seem to feed it the right form values to get the data out of
it.  Heres the code:

import urllib
import urllib2

## The source page http://www.erh.noaa.gov/bgm/climate/bgm.shtml
url = 'http://www.erh.noaa.gov/bgm/climate/pick.php'
values = {'month' : 'July',
          'year' : '1988'}

user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
headers = { 'User-Agent' : user_agent }

data = urllib.urlencode(values)
req = urllib2.Request(url, data, headers)
response = urllib2.urlopen(req)
the_page = response.read()
print the_page



More information about the Python-list mailing list