[Python-Dev] PEP 476: Enabling certificate validation by default!

[Nick Coghlan]

On 31 August 2014 12:21, R. David Murray <rdmurray at bitdance.com> wrote:
> Do those knobs allow one to instruct urllib to accept an invalid
> certificate without changing the program code?

My first reply ended up being a context dump of the challenges created
by legacy corporate intranets that may not be immediately obvious to
folks that spend most of their time working on or with the public
internet. I decided to split these more technical details out to a new
reply for the benefit of folks that already know all that history :)

To answer David's specific question, the existing knobs at the OpenSSL
level (SSL_CERT_DIR and SSL_CERT_FILE ) let people add an internal CA,
opt out of the default CA system, and trust *specific* self-signed
certs.

What they don't allow is a global "trust any cert" setting -
exceptions need to be added at the individual cert level or at the CA
level, or the application needs to offer an option to not do cert
validation at all. That "trust anything" option at the platform level
is the setting that is a really bad idea - if an organisation thinks
it needs that (because they have a lot of self-signed certs, but
aren't verifying their HTTPS connections to those servers), then what
they really need is an internal CA, where their systems just need to
be set up to trust the internal CA in addition to the platform CA
certs.

With Alex's proposal, organisations that are already running an
internal CA should be just fine - Python 3.5 will see the CA cert in
the platform cert store and accept certs signed by it as valid. (Note:
the Python 3.4 warning should take this into account, which could be a
problem since we don't currently do validity checks against the
platform store by default. The PEP needs to cover the mechanics of
that in more detail, as I think it means we'll need to make *some*
changes to the default configuration even in Python 3.4 to get
accurate validity data back from OpenSSL)

However, we also need to accept that there's a reason browser vendors
still offer "click through insecurity" for sites with self-signed
certificates, and tools like wget/curl offer the option to say "don't
check the certificate": these are necessary compromises to make SSL
based network connections actually work on many current corporate
intranets.

It is corporate environments that also make it desirable to be able to
address this potential problem at a *user* level, since many Python
users in a large organisations are actually running Python entirely
out of their home directories, rather than as a system installation
(they may not even have admin access to their own systems).

My suggestion at this point is that we take a leaf from both browser
vendors and the design of SSH: make it easy to *add* a specific
self-signed cert to the set a *particular user* trusts by default
(preferably *only* for a particular host, to limit the power of such
certs). "python -m ssl" doesn't currently do anything interesting, so
it could be used to provide an API for managing that user level
certificate store.

A Python-specific user level cert store is something that could be
developed as a PyPI library for Python 2.7.9+ and 3.4+ (Is cert
management considered in scope for cryptography.io? If so, that could
be a good home).

So while I agree with the intent of PEP 476, and like the suggested
end state, I'm back to thinking that the transition plan for existing
corporate users needs more work before it can be accepted. This is
especially true since it becomes another barrier to migrating from
Python 2.7 to Python 3.5+ (a warning in Python 3.4 doesn't help with
that aspect, although a new -3 warning might).

A third party module that offers a user level certificate store, and a
gevent.monkey style way of opting in to this behaviour for existing
Python versions would be one way to provide a more compelling
transition plan.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia