cgi POST question

[Steve Holden]

"Christopher" <chris_mk at hotmail.com> wrote ...
> Hi,
>      I am relatively new to programming in general and Python in
> particular (Two years programming basic stuff and the last six months
> with Python).  For the most part, I use Python purely to process and
> extrapolate data.  I have absolutely no Network Programming
> experience.  Lately, I have been using the urllib module to construct
> cgi queries on the fly (get a list of keywords, etc. and plug them
> into a generic cgi query).  As an example (not real code):
>
> ListOfQueries = []
> cgiQuery =
"http://www.site.com/query.cgi?keyword=MYWORD&defaults=blahblah"
> for x in [MyListOfStuff]:
>         newQuery = cgiQuery.replace('MYWORD', x)
>         ListOfQueries.append(newQuery)
>
>
> I would then use the formatted queries for whatever.  My boss asked me
> to add  the ability to do the same on our intranet server.  I thought
> it would be easy.  I was wrong.  Here is what I think is going on
> (though I am sure I have some details wrong).
>      A query like the one above is what is passed when a method of
> 'GET' is specified (the intranet search specifies a method of 'POST'
> instead, I realized much later).  I carefully constructed the cgi
> query, but it wasn't returning the correct results.  I then took the
> source itselt, modified it to submit using a method of 'GET' and
> tried.  The cgi query that was produced matched what I had constructed
> but still no results.  I spoke with the guy in charge of the intranet
> search engine and he said that the cgi script he wrote is incompatible
> with a 'GET' method (my ignorance shows itself, I thought it didn't
> matter which method was used) and he doesn't have the time or the
> inclination to change it (he was also a little bitter that I would
> even be 'messing' with his 'stuff' even though it was his boss' boss
> that told me to do it).  I looked at the cgi module documentation, but
> it doesn't seem to be able to do what I want.

You are writing client-side programs, and wanting to retrieve data from the
intranet web server, correct? The urllib library *is* the one you want ...
see below.

As far as the intranet search engine guy goes, his behavior betrays a
fragile ego and low self-esteem typical of people who aren't really
comfortable doing what they are doing. The good news is you can ignore hom
from here on in.

It's likely he *could* have written his code to work with *either* POST or
GET, but many people don't know the appropriate techniques. Ironically, in
ASP and Cold Fusion it's actually *easier* to write CGI scripts this way!

>      What I want, by the way, is to know if there is a python module
> that submits information in the 'POST' method.  With urllib, I can
> submit information in the 'GET' method (all it is is a website, after
> all), but I can't figure out how to do the same with the POST method
> (I was thinking it would be an object, with attributes in a dictionary
> and a method to post it to the cgi script in the correct format,
> soemthing like:
>
> myTest = cgimodule.cgiPostObject
> myTest['keyword'] = "MyWord"
> myTest['keyword2'] = "MyOtherWord"
> myTest.cgiSubmit("http://www.site.com/query.cgi", method = 'POST')
>
>
> or something to that effect).  I would write it myself, but I wouldn't
> know how.

You are so close you clearly understand Python quite well. See below ...

>      I did have my boss approve the purchases of a couple networking
> books and "Python Web Programming" by Steve Holden (<= very good so

... and you a clearly a person of discrimination :-) Glad you're enjoying
the book ...

> far, btw) but I am still not up to speed yet (basically, I am learning
> Networking stuff, but reading at home, since I am so busy at work).  I
> really need to figure this out because patience is relatively low
> right now.  Any help would be appreciated.  Thank you very much.  I
> would be happy to answer any additional questions. Talk to you all
> later.
>
Here's an exceprt from the urllib docs:

"""
urlopen (url[, data])
Open a network object denoted by a URL for reading. If the URL does not have
a scheme identifier, or if it has file: as its scheme identifier, this opens
a local file; otherwise it opens a socket to a server somewhere on the
network. If the connection cannot be made, or if the server returns an error
code, the IOError exception is raised. If all went well, a file-like object
is returned. This supports the following methods: read(), readline(),
readlines(), fileno(), close(), info() and geturl().
...
If the url uses the http: scheme identifier, the optional data argument may
be given to specify a POST request (normally the request type is GET). The
data argument must in standard application/x-www-form-urlencoded format; see
the urlencode() function below.
"""

In other words, as you surmised, you can fill a dictionary with the data you
want to POST. If you pass the dictionary as a second argument to
urllib.urlopen(), you do a POST.

I'd write you some sample code, except you sound like you're quite capable
of taking it from here. Let us know how it works out.

regards
 Steve
--
Consulting, training, speaking: http://www.holdenweb.com/
Author, Python Web Programming: http://pydish.holdenweb.com/pwp/

"This is Python.  We don't care much about theory, except where it
intersects with useful practice."  Aahz Maruch on c.l.py