Searching Google?

Alex metallourlante at gmail.com
Wed Feb 18 10:16:10 EST 2009


On Feb 18, 1:36 pm, Johannes Bauer <dfnsonfsdu... at gmx.de> wrote:
> Curt Hash schrieb:
>
> > You just need to change your User-Agent so that Google doesn't know a
> > Python script is making the request:
>
> Why would Google not send a response if the User-Agent header field is
> not a recognized browser?

Because making automated queries to Google is against its TOS so
Google block any client that doesn't seam to be human.
On the other hands Google's API does not return the same exact result
as in a normal web query.
My suggestion is: set a browser like User agent and accept gzipped
content to be as friendly as possible and don't do too many queries in
a small time span.
Parsing Google result page with Beautifulsoup is a piece of cake.

Alex



More information about the Python-list mailing list