[Tutor] Beautifulsoup Queries

Crusier crusier at gmail.com
Tue Dec 8 21:01:59 EST 2015


Dear All,

I am using Python 3.4, I tried to scrap the web and eventually put
those data into a database for analysis. While I am using
Beautifulsoup to scrap the web, I encountered 2 problems:

1. Using Beautiful Soup, the webmaster on the other end is using the
same class, so I got a whole list of information (High, Low Previous
Close, Shares Traded, Turnover.....etc) and I really want to divide
that up into separate categories.

2. If I input  " print(data.string) " for line 27, I will get an error
('ResultSet' object has no attribute 'string'). However, if I input  "
print(data) ", they print out in span class..... etc...

from bs4 import BeautifulSoup
import requests
import os


tryout = ['0001', '0002', '0003', '0004', '0005', '0006', '0007',
'0008', '0009', '0010', '0011', '0012', '0014', '0015', '0016',
'0017', '0018', '0019', '0020']

url = "https://www.etnet.com.hk/www/eng/stocks/realtime/quote.php?code="



def web_scraper(url):
    for n in tryout:
        URL = url + n
        response = requests.get(URL)
        html = response.content
        soup = BeautifulSoup(html,"html.parser")
        # print (soup.prettify())

        Title = soup.find("div", attrs = {"id": "StkQuoteHeader"})
        RT_down = soup.find("span", attrs = {"class": "Price down2"})
        RT_up = soup.find("span", attrs = {"class": "Price up2"})
        RT_unchange =  soup.find("span",attrs = {"class" :"Price unchange2"})
        change_percent = soup.find("span", attrs = {"class" :"Change"})
        Day_High = soup.findAll("span", attrs = {"class" :"Number"})
        for data in [Title, RT_down, RT_up, RT_unchange,
change_percent, Day_High]:
            if data:
                print(data)


web_scraper(url)


More information about the Tutor mailing list