Searching Google?

[Tim Wintle]

On Wed, 2009-02-18 at 13:36 +0100, Johannes Bauer wrote:
> Curt Hash schrieb:
> 
> > You just need to change your User-Agent so that Google doesn't know a
> > Python script is making the request:
> 
> Why would Google not send a response if the User-Agent header field is
> not a recognized browser? I doubt that's what's happening and the OP
> would just like the convenience of a search API instead of parsing the
> web page (like other applications do). Ugly, but it works.

I suspect that it's the other way around - Google has black-listed the
standard python user-agent rather than whitelisting useragents.

Think about how much power it takes to do a query on Google - if they
provided a search API they would lose out on advertising on the results
- which at the end of the day is their income-source.

It's a pain not to have a search API, but you've got to understand it!