stealth screen scraping with python?

Thomas Wittek mail at gedankenkonstrukt.de
Sat May 12 05:34:16 EDT 2007


different.engine at gmail.com schrieb:
> I am screen scraping a large volume of data from Yahoo Finance each
> evening, and parsing with Beautiful Soup.
> 
> I was wondering if anyone could give me some pointers on how to make
> it less obvious to Yahoo that this is what I am doing, as I fear that
> they probably monitor for this type of activity, and will soon ban my
> IP.

Use anonymizing proxies:
http://www.google.com/search?q=proxies+OR+proxy+anonymous+OR+anonymizing

-- 
Thomas Wittek
http://gedankenkonstrukt.de/
Jabber: streawkceur at jabber.i-pobox.net



More information about the Python-list mailing list