urllib/urllib2 help

[Steve Holden]

"Gabe Newcomb" <Gabe.Newcomb at noetix.com> wrote ...

> I'm doing some automated testing of an API that can be called by http
> requests (I'm using POST). For these purposes, I'm using urllib2, which
> is working fabulously...except for one thing. Some of the calls require
> a browser session to be in existence so that they can keep referring to
> it. I'm fairly ignorant of how all of this works, but the one thing I
> know is that using urllib2.urlopen() does not create any type of
> session.
>
Browser sessions are created using cookies. The server sends your browser
(or, as you are now discovering, other HTTP client) a response which
includes one or more Ste-Cookie: headers. The cookies are specific to the
server's domain (or some selected subdomain, such as the server itself) and
a content-path (such as / to cover the whole site, or /secret/ to cover only
URLs in that subdirectory).

Whenever the client makes a request it should examine its cookie jar (where
it stores the cookies that have been sent by the various servers it has
interacted with) and include (as Cookie: headers) those that match the
domain and path of the URL it is requesting.

This is a very general mechanism: you can get more background at

    http://www.netscape.com/newsref/std/cookie_spec.html

but, of course, you'll have to do a little digging to find out how the
server you are using expects the browser to maintain state. Just
"iomplementing" cookies is usually enough, but there are wrinkles when you
get to things like HTTP redirections.

> My question is this: is there any way for me to create a session object
> for my testing? Would I have to work with IE's object model or
> something? Is there a win32* library for this?
>
There has been some discussion recently about this, but I'm not sure anyone
is yet hacking code for the standard library. The standard library includes
a Cookie module that you can use to generate cookie headers in your HTTP,
but I'm not familiar enough with urllib2 to understand how you can insert
thm as request headers, which is what you need to do.

> I hope this question is on-topic enough. I love doing my testing in
> Python, and I hope I can find a way to keep using it for this effort.
>
Certainly not off-topic, and as you are beginning to realise there's almost
nothing you *can't* do in Python. Let us know when you get it to work!

regards
 Steve
--
Steve Holden: http://www.holdenweb.com/ ; Python Web Programming:
http://pydish.holdenweb.com/pwp/