Problem when scraping the 100 Movie titles.

Bischoop Bischoop at vimart.net
Thu Feb 18 10:22:08 EST 2021


I'm learning Scraping actually and would like to scrape the movie titles
from https://www.empireonline.com/movies/features/best-movies-2 . 
In the course I was learning I was supposed to do it with bs4:
titles = soup.find_all(name = 'h3', class_ = 'title')

but after after a while I guess the site has changed and now the class
is: jsx-2692754980

<h3 class="jsx-2692754980">100) Stand By Me</h3>

but anyway if I do try get those titles by name and class, my list is
empty:
titles = soup.find_all(name = 'h3', class_ = 'jsx-2692754980')

I tried also selenium and manage get those titles with:
driver.get('https://www.empireonline.com/movies/features/best-movies-2')
#driver.find_element_by_xpath('/html/body/div/div[3]/div[5]/button[2]').click()

titles = driver.find_elements_by_css_selector("h3.jsx-2692754980")

tit=[]
for e in titles:
    tit.append(e.text)

    print(tit)

But in Chrome I get a popup asking to accept cookies and I need to
click to accept them.

Is someone here who knows how can I get those titles with BeautifulSoup and how to deal with
cookies if using Selenium?

--
Thanks


More information about the Python-list mailing list