How can I export data from a website and write the contents to a text file?

ryguy7272 ryanshuell at gmail.com
Wed Nov 18 12:43:59 EST 2015


On Wednesday, November 18, 2015 at 12:41:19 PM UTC-5, ryguy7272 wrote:
> On Wednesday, November 18, 2015 at 12:21:47 PM UTC-5, Denis McMahon wrote:
> > On Wed, 18 Nov 2015 08:37:47 -0800, ryguy7272 wrote:
> > 
> > > I'm trying the script below...
> > 
> > The problem isn't that you're over-writing the lines (although it may 
> > seem that way to you), the problem is that you're overwriting the whole 
> > file every time you write a link to it. This is because you open and 
> > close the file for every link you write, and you do so in file mode "wb" 
> > which restarts writing at the first byte of the file every time.
> > 
> > You only need to open and close the text file once, instead of for every 
> > link you output. Try moving the lines to open and close the file outside 
> > the outer for loop to change the loop from:
> > 
> > for item in soup.find_all(class_='lister-list'):
> >     for link in item.find_all('a'):
> >         # open file
> >         # write link to file
> >         # close file
> > 
> > to:
> > 
> > # open file
> > for item in soup.find_all(class_='lister-list'):
> >     for link in item.find_all('a'):
> >         # write link to file
> > # close file
> > 
> > Alternatively, use the with form:
> > 
> > with open("blah","wb") as text_file:
> >     for item in soup.find_all(class_='lister-list'):
> >         for link in item.find_all('a'):
> >             # write link to file
> > 
> > -- 
> > Denis McMahon, 
> 
> 
> Yes, I just figured it out.  Thanks.  
> 
> It doesn't seem like the '\n' is doing anything useful.  All the text is jumbled together.  When I open the file in Excel, or Notepad++, it is easy to read.  However, when I open it in as a regular text file, everything is jumbled together.  Is there an easy way to fix this?

I finally got it working.  It's like this:
"\r\n"

Thanks everyone!!



More information about the Python-list mailing list