Automatically populate and submit HTML Forms

[Grant Edwards]

On 2005-05-25, rbt <rbt at athop1.ath.vt.edu> wrote:
> How can I make a python client script connect to a Web server and 
> automatically populate form fields and then submit the form?
>
> For example, say I wanted to check and see if 1924 was a leap year... 
> how would I populate the 'year' field and then submit it for processing? 
> I have no control of the server... just the client script.
>
><form action="cgi-bin/leapyear.py">
>  <p>Enter a year and find out if it's a leap year:
>  <input type="text" name="year" size="6">
>  <input type="submit">
>  <input type="reset">
></form>

Just use urllib() and pass the form data to the urlopen()
method.  If given data, it will generate a "POST" request
instead of a "GET".  Here's a snippet of code from an app of
mine that "fills in a form" and submits it:

    postData = urllib.urlencode({'submit':'Remove','disp':'M','action':'change_Msgs'})
    
    for msgid in msgIDs:
        postData += "&msgid="+msgid

    req2 = urllib2.Request("http://mc-s6.postini.com/exec/MsgCtr",postData)
    rsp2 = ClientCookie.urlopen(req2)

In this code I've eyeballed the form and the field names are
hard-wired into the code.  If your form doesn't change from one
usage to the next, that's the simplest way to do it.

In my example I'm using ClientCookie and urllib2 to create/open
the reqeust in two steps because the request seen above won't
work without some cookie values previsouly established in code
that I've snipped.  Otherwise all you'd need to do is something
like this:

  urllib.urlopen('http://whatever',
                  urllib.urlencode({'field1Name':'value1','field2Name':'value2'}))
                  
-- 
Grant Edwards                   grante             Yow!  Yes, but will I
                                  at               see the EASTER BUNNY in
                               visi.com            skintight leather at an
                                                   IRON MAIDEN concert?