https proxy

[Paul Sweeney]

Paul Rubin wrote:
> "Paul Sweeney" <reverse.ku.oc.issolok at nothypgnal.delrest.co.uk> writes:
> > I wish to access an https:// site using my browser and click around
> > it.  I wish to see the http(s) GET urls and http(s) POST urls and
> > data which are being sent to the https server.  If the site were an
> > http:// site I would use a tool like ethereal to see what was going
> > on, but this data is sent encrypted from the browser due to the
> > https connection, so I'll just see the encrypted data. So I need a
> > tool to view the unencrypted form of the data which is being sent
> > encrypted (sorry for being unclear about the "unecrypted data" in my
> > last post, I hope this is better).
>
> You could do that but I don't know of already-existing Python code
> that does it.

Rats, that would seem to be the consensus on this, the nearest is the http
proxies which allow https to tunnel through them leaving the data encrypted.
At least they would do if this feature of any of them actually worked ;-)

> Basically it would be a man-in-the-middle attack against
> the browser/server pair:
>
>   browser---(SSL)---proxy---(SSL)---server
>
> There'd be two separate SSL sessions and the proxy would of course
> have to present its own server certificate to the browser, which you'd
> configure the browser to accept.  The server application could take
> various measures to detect this subterfuge and stop it from happening,
> but I doubt any normal server apps actually bother to do that.  It's
> not a true security break, since the browser user has to accept the
> proxy certificate, thereby giving "permission" for the interception.

A perfect description of the problem and possible solutions, thanks Paul,
glad you could make this out from my descriptions :-D

This is exactly what the java Paros tool does, but I'd like to have a Python
version which I can customise to do the things I need automatically.  I'm
working on an automatic testing framework written in python which can
perform scripted actions on a target system, and want an easy way to
determine the urls of GETs and the urls (including query strong) and data of
POSTS performed under https connections. In order to replay them later.  I'm
doing it with the target server owners permission, but don't have access to
fiddle with server configuration settings etc.

Just to emphasise again, I'm only trying to see the data that my browser is
sending/receiving, *not* intercept someone elses connection.

Any posts for suggested python software gratefully received.  If I end up
writing one of these, I'll try to get agreement from my employer for it to
be open sourced since there seems to be an opening for such a tool, even if
there isn't much of a market for it!