[Tutor] Python Scraping problem help
CPECC张涛
sccdzt at foxmail.com
Thu May 27 01:38:39 EDT 2021
Dear sir :
Could you please help solve the below problem that can run but no result ,the program is runing,but can noly retrieve one picture ,Could you help tell me how to solve this problem .
import requests
from bs4 import BeautifulSoup
import urllib
import time,os
headers={'User-Agent': 'Mozilla/5.0 (Windows NT 6.1;'
'Win64; x64) AppleWebKit/537.36 (KHTML, like'
'Gecko) Chrome/69.0.3497.100'
'Safari/537.36','Referer': "https://www.douban.com/group/topic/206388806/?type=rec"}
url=r'https://www.douban.com/group/topic/206388806/?type=rec'
wbdata=requests.get(url,headers=headers)
# print(response)
soup=BeautifulSoup(wbdata.content,'lxml')
imgs=soup.select('div.image-wrapper>img')
# print(imgs)
number=1
for item in imgs:
img=item.get('src')
print(img)
path=r'F:\Python crawler'
if not os.path.isdir(path):
os.makedirs(path)
paths=path+str(number)+'.jpg'
time.sleep(1)
try:
urllib.request.urlretrieve(img,paths)
number+=1
# print('%sDownloaded'%img)
except:
continue
More information about the Tutor
mailing list