[Python-Dev] Verification of SSL cert and hostname made easy

[Christian Heimes]

Hi,

Larry has granted me a special pardon to add an outstanding fix for SSL,
http://bugs.python.org/issue19509 . Right now most stdlib modules
(ftplib, imaplib, nntplib, poplib, smtplib) neither support server name
indication (SNI) nor check the subject name of the peer's certificate
properly. The second issue is a major loop-hole because it allows
man-in-the-middle attack despite CERT_REQUIRED.

With CERT_REQUIRED OpenSSL verifies that the peer's certificate is
directly or indirectly signed by a trusted root certification authority.
With Python 3.4 the ssl module is able to use/load the system's trusted
root certs on all major systems (Linux, Mac, BSD, Windows). On Linux and
BSD it requires a properly configured system openssl to locate the root
certs. This usually works out of the box. On Mac Apple's openssl build
is able to use the keychain API of OSX. I have added code for Windows'
system store.

SSL socket code usually looks like this:

  context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
  context.verify_mode = ssl.CERT_REQUIRED
  # new, by default it loads certs trusted for Purpose.SERVER_AUTH
  context.load_default_certs()

  sock = socket.create_connection(("example.net", 443))
  sslsock = context.wrap_socket(sock)

SSLContext.wrap_socket() wraps an ordinary socket into a SSLSocket. With
verify_mode = CERT_REQUIRED OpenSSL ensures that the peer's SSL
certificate is signed by a trusted root CA. In this example one very
important step is missing. The peer may return *ANY* signed certificate
for *ANY* hostname. These lines do NOT check that the certificate's
information match "example.net". An attacker can use any arbitrary
certificate (e.g. for "www.evil.net"), get it signed and abuse it for
MitM attacks on "mail.python.org".
http://docs.python.org/3/library/ssl.html#ssl.match_hostname must be
used to verify the cert. It's easy to forget it...


I have thought about multiple ways to fix the issue. At first I added a
new argument "check_hostname" to all affected modules and implemented
the check manually. For every module I had to modify several places for
SSL and STARTTLS and add / change about 10 lines. The extra lines are
required to properly shutdown and close the connection when the cert
doesn't match the hostname. I don't like the solution because it's
tedious. Every 3rd party author has to copy the same code, too.

Then I came up with a better solution:

  context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
  context.verify_mode = ssl.CERT_REQUIRED
  context.load_default_certs()
  context.check_hostname = True  # <-- NEW

  sock = socket.create_connection(("example.net", 443))
  # server_hostname is already used for SNI
  sslsock = context.wrap_socket(sock, server_hostname="example.net")


This fix requires only a new SSLContext attribute and a small
modification to SSLSocket.do_handshake():

  if self.context.check_hostname:
      try:
          match_hostname(self.getpeercert(), self.server_hostname)
      except Exception:
          self.shutdown(_SHUT_RDWR)
          self.close()
          raise


Pros:

* match_hostname() is done in one central place
* the cert is matched as early as possible
* no extra arguments for APIs, a context object is enough
* library developers just have to add server_hostname to get SNI and
hostname checks at the same time
* users of libraries can configure cert verification and checking on the
same object
* missing checks will not pass silently

Cons:

* Doesn't work with OpenSSL < 0.9.8f (released 2007) because older
versions lack SNI support. The ssl module raises an exception for
server_hostname if SNI is not supported.


The default settings for all stdlib modules will still be verify_mode =
CERT_NONE and check_hostname = False for maximum backward compatibility.
Python 3.4 comes with a new function ssl.create_default_context() that
returns a new context with best practice settings and loaded root CA
certs. The settings are TLS 1.0, no weak and insecure ciphers (no MD5,
no RC4), no compression (CRIME attack), CERT_REQUIRED and check_hostname
= True (for client side only).

http://bugs.python.org/issue19509 has a working patch for ftplib.

Comments?

Christian