Error getting data from website

Michael Torrie torriem at gmail.com
Fri Dec 6 19:38:18 EST 2019


On 12/6/19 5:31 PM, DL Neil via Python-list wrote:
> If you read the HTML data that the REPL has happily splattered all over 
> your terminal's screen (scroll back) (NB "soup" is easier to read than 
> is "content"!) you will observe that what you saw in your web-browser is 
> not what Amazon served in response to the Python "requests.get()"!

Sadly it's likely that Amazon's page is largely built from javascript.
So scraping static html is probably not going to get you where you want
to go.  There are heavier tools, such as Selenium that uses a real
browser to grab a page, and the result of that you can parse and search
perhaps.


More information about the Python-list mailing list