https proxy

[Peter Hansen]

Paul Sweeney wrote:

> I wish to access an https:// site using my browser and click around it.  I
> wish to see the http(s) GET urls and http(s) POST urls and data which are
> being sent to the https server.  If the site were an http:// site I would
> use a tool like ethereal to see what was going on, but this data is sent
> encrypted from the browser due to the https connection, so I'll just see the
> encrypted data. So I need a tool to view the unencrypted form of the data
> which is being sent encrypted (sorry for being unclear about the "unecrypted
> data" in my last post, I hope this is better).

Based on what you say here, I think the answer is back to being
"impossible".  The encryption is going on in the browser.  If
the browser thinks it's talking HTTPS, then it will encrypt
stuff and talk HTTPS.  Nothing between the browser end of the
socket and the server end of the socket will be able to
decrypt it.

If you had a proxy that used its own certificate and established
an https connection when the client tried to connect with http
to something, then you could at least start the whole thing off
without encrypted data on the browser side.  As soon as you tried
to click on an embedded https link, however, the browser would
try to establish an encrypted link, and the proxy wouldn't work
any more.

What about just writing a tool using urllib2 and basically
simulating the browser activity in Python?  Then you can
bypass the whole issue, and have programmatic access to the
unencrypted data before or after it hits the encrypted socket
(depending on the direction of travel).

> I have and could use a tool like Paros, but it is java and (need I say
> more?):
> 
> a) I love Python (:-D) ...
> b) ... seriously, I'd like to be able to play with the source code to record
> the gets and posts for later replay in a python based retriever tool.

I love Python too, but that doesn't stop me from using a more
effective tool when the need arises.  Surely something like Paros,
if it can do what you need (and I don't see how it can), also
has the ability to save the data to a file for later perusal,
even if the utility for retrieving it were in Python.

-Peter