tracing openurl input and output?

[Duncan Booth]

mh at pixar.com wrote:

> How can I watch the messages being sent back and for on urllib shttp
> requests?  If it were simple http I would just watch the socket traffic
> but of course that won't work for https.  Is there a debug flag I can
> set that will do this?
> 
> context:  I am dealing with a web service bug and I want to tell
> the provider exactly what is going back and forth to his server,
> eliminating the "maybe there's a bug in your library" chat.
> 
>     import urllib
>     params = urllib.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
>     f = urllib.urlopen("https://example.com/cgi-bin/query", params)
> 
> Many TIA!
> 
If you are running on Windows then I'd say get hold of a copy of fiddler2 
(http://www.fiddler2.com/fiddler2/) and tell your code to use it as a 
proxy. It does an excellent job of capturing and decoding web traffic and 
even lets you modify packets as they go past. The only thing to watch for 
with https is that your code will see an invalid certificate as of course 
it sees fiddler's own self signed certificate instead of the original.

I'm not sure what Linux or Mac equivalents there might be (though in a 
pinch you can always run your code on one system and Fiddler on a different 
one).