How can I export data from a website and write the contents to a text file?

ryguy7272 ryanshuell at gmail.com
Wed Nov 18 11:37:47 EST 2015


I'm trying the script below, and it simple writes the last line to a text file.  I want to add a '\n' after each line is written, so I don't overwrite all the lines.


from bs4 import BeautifulSoup
import urllib2

var_file = urllib2.urlopen("http://www.imdb.com/chart/top")

var_html  = var_file.read()

var_file.close()
soup = BeautifulSoup(var_html)
for item in soup.find_all(class_='lister-list'):
    for link in item.find_all('a'):
        print(link)
        text_file = open("C:/Users/rshuell001/Desktop/excel/Text1.txt", "wb")
        z = str(link)
        text_file.write(z + "\n")
        text_file.write("\n")
        text_file.close()


Can someone please help me get this working?
Thanks!!



More information about the Python-list mailing list