urllib(2) and https blues? try pytunnel for python tunnelling

[john]

The python libraries like urllib and httplib do not support ssl
through a proxy.
Urllib2 supports http through a proxy or https alone, but not https
through a proxy.

A while ago my wife complained to me that perl had trouble w/https. I
was surprised since perl has a lot of networking libraries. Preferring
python, I took a look a python and found the same problem. Eventually
I figured out what python was missing: tunnelling. Though she may
_coax_ me to port the code to perl, I'm writing it first in my
favorite language.

Pytunnel offers a tunnel that can be used for (among other things)
tunnelling the ssl through a proxy.

I'd be willing to offer suggestions or myself add the necessary code
to urllib2. Anyone know how one goes about doing either?

Your python code does not need to be changed except that the port and
ip are supplied by the tunnel. Pytunnel uses a custom recvall which is
like sendall but for recv. Recvall uses non-blocking sockets,timeouts
and sleeps to attempt to get all of the data. If you have a bad
network connection you'd want to set the timeout to be high. Recvall
will probably be slower than a standard recvall.

The code is rough and needs better error handling, comments and such.
But since I got it working I decided to post it.

You can find it at:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/213238
Example of usage:

import pytunnel,httplib

def tunnel_this(ip,port):
   conn = httplib.HTTPSConnection(ip,port=port)
   conn.putrequest('GET', '/')
   conn.endheaders()
   response = conn.getresponse()
   print response.read()

tunnel=pytunnel.build(host='login.yahoo.com',proxy_host='h1',proxy_user='u',proxy_pass='p')
tunnel.run(tunnel_this)