accessing a web search engine

[Paul Rubin]

Rajarshi Guha <rxg218 at psu.edu> writes:
> Hi,
>   I'm trying to use Python to send queries to a webpage and analyse the 
> results. However the search engine does'nt use cgi scripts - it uses a 
> Javascript function. How can I access a JScript function. I realize that 
> this is not really a Python question, but since I am using it I thought I'd 
> ask here. I know that I have to use the urllib module but what string do I 
> send to the server?

You have to analyze the javascript on the web page and figure out what
it's really doing when you click the search button.  Sooner or later
it probably does a form.submit() call.  Find the target attribute for
that form and navigate your client there with the appropriate query or
POST arguments.  If the page is doing a nontrivial javascript
computation before submitting, you'll have to do a similar calculation
in your python client.