Using search fields on websites

[Ian Bicking]

I would recommend using urllib/urllib2 to retrieve the web pages (using
just the open function).  If you aren't familiar with how forms work,
you can probably figure out by example by looking at the URL you get
from various searches -- in most search forms all the information you
entered in the form gets passed through the URL.  (If you don't see your
search terms in the URL, you are probably using a POST form -- save the
page and replace the POST in 
<form action="url" method="POST"> with GET, and replace the url with an
absolute url, and try again).  

You should use urllib.quote to quote your search terms (since, for
instance, you can't have a space in a URL -- urllib.quote will encode
that space properly).



On Wed, 2002-07-17 at 01:25, Jen wrote:
> Hi All,
> 
> I'm trying to use python to do searches on a website by entering a
> string into a search field to search against the website's database.
> 
> Currently I have a list of strings which I want to automate to enter
> each into a textfield on a website, select a different radio button
> from the default and submit for searching. Is there a way that I can
> do this easily?
> 
> If anyone could suggest not only the modules to use but also any
> possible functions within the modules, that would be greatly
> appreciated. Thanks.
> 
> Jen
> -- 
> http://mail.python.org/mailman/listinfo/python-list