How can I export data from a website and write the contents to a text file?
ryguy7272
ryanshuell at gmail.com
Wed Nov 18 12:43:59 EST 2015
On Wednesday, November 18, 2015 at 12:41:19 PM UTC-5, ryguy7272 wrote:
> On Wednesday, November 18, 2015 at 12:21:47 PM UTC-5, Denis McMahon wrote:
> > On Wed, 18 Nov 2015 08:37:47 -0800, ryguy7272 wrote:
> >
> > > I'm trying the script below...
> >
> > The problem isn't that you're over-writing the lines (although it may
> > seem that way to you), the problem is that you're overwriting the whole
> > file every time you write a link to it. This is because you open and
> > close the file for every link you write, and you do so in file mode "wb"
> > which restarts writing at the first byte of the file every time.
> >
> > You only need to open and close the text file once, instead of for every
> > link you output. Try moving the lines to open and close the file outside
> > the outer for loop to change the loop from:
> >
> > for item in soup.find_all(class_='lister-list'):
> > for link in item.find_all('a'):
> > # open file
> > # write link to file
> > # close file
> >
> > to:
> >
> > # open file
> > for item in soup.find_all(class_='lister-list'):
> > for link in item.find_all('a'):
> > # write link to file
> > # close file
> >
> > Alternatively, use the with form:
> >
> > with open("blah","wb") as text_file:
> > for item in soup.find_all(class_='lister-list'):
> > for link in item.find_all('a'):
> > # write link to file
> >
> > --
> > Denis McMahon,
>
>
> Yes, I just figured it out. Thanks.
>
> It doesn't seem like the '\n' is doing anything useful. All the text is jumbled together. When I open the file in Excel, or Notepad++, it is easy to read. However, when I open it in as a regular text file, everything is jumbled together. Is there an easy way to fix this?
I finally got it working. It's like this:
"\r\n"
Thanks everyone!!
More information about the Python-list
mailing list