urllib2/httplib and email mime

[John J. Lee]

someone at arbitrary.org (Joseph) writes:

> I am using email to create a multipart message that I need to post to
> a url.
> 
> I have been doing it semi-manually with httplib:
[...]
> Can someone sugest a cleaner way of doing this sort of thing using
> urllib2? The problem is getting the Content-Type header from my email
> object to be used by urllib2 as an http header.

http://wwwsearch.sourceforge.net/ClientForm/src/README-0_1_7b.html

(note only 0.1.x versions do file upload, not 0.0.x)

I haven't added file upload to the long example on that page, so
here's one:

import urllib2, ClientForm
# url here is URL of web page containing the form you're submitting
response = urllib2.urlopen(url)
forms = ClientForm.ParseResponse(response)
response.close()

form = form[0]

for name in filenames:
    f = file(filename)  # use StringIO if you have a string to upload
    # mimetype and filename are optional
    form.add_file(f, "text/plain", "upload.txt")

# (do other form-filling if necessary)

request = form.click()
response2 = urllib2.urlopen(request)
response2.read()  # body
response2.info()  # headers


You can cache the form (in a class, on on disk with pickle) or the
HTML (ClientForm.ParseFile is useful for that) if you don't want to
fetch the form page every time you submit a form.

A minor wart I just noticed: ClientForm only reads from the files when
you actually do form.click(), so the code above relies on Python to
close the file objects you're uploading from (not a problem in
CPython).  Also, I haven't yet tested multifile upload on anything but
Python's cgi module -- but I'm likely to fix it quickly if it doesn't
work.  And I'm sure my MIME code isn't as sophisticated as the email
module -- for example, it doesn't yet guess MIME types.


HTH


John