[Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

[Nick Coghlan]

On 3 June 2017 at 08:21, Nathaniel Smith <njs at pobox.com> wrote:
> On Fri, Jun 2, 2017 at 1:29 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> On 6/2/2017 12:21 PM, Barry Warsaw wrote:
>>>
>>> On Jun 03, 2017, at 02:10 AM, Nick Coghlan wrote:
>>
>>
>>>> The benefit of making any backport a private API is that it would mean
>>>> we weren't committing to support that API for general use: it would be
>>>> supported *solely* for the use case discussed in the PEP (i.e. helping
>>>> to advance the development of PEP 543 without breaking pip
>>>> bootstrapping in the process).
>>>
>>>
>>> That sounds like a good compromise.  My own major objection was in
>>> exposing a
>>> new public API in Python 2.7, which would clearly be a new feature.
>>
>>
>> Which would likely be seen by someone as justifying other requests to add to
>> 2.7 'just this one more essential new feature' ;-).
>
> Whatever the eventual outcome, I don't think there's any danger
> someone will read this thread and think "wow, it's so easy to get new
> features into 2.7".

Aye, when "Be related to the Python ecosystem's collective handling of
one of the essential network protocols that underpins the security of
the entire internet" is a criterion to even *start* the backport
discussion (let alone successfully make the case), I don't think "We
might get deluged with further backport proposals" is really something
we need to worry about :)

Another key factor is being able to convince folks with some level of
influence with commercial redistributors that the idea is worth
supporting - the upstream backport in CPython is only step one in
rolling out this kind of change, and we need to get it all the way
through into the redistributor channels for it to really have the
desired impact (hence PEP 493 as a follow-up to PEP 476 - at least
some redistributors, including Red Hat, said "No" to the original hard
compatibility break, but were OK with a smoother transition plan that
gave end users more control over the timing of the change in
behaviour).

For the MemoryBIO and SSLObject backport, I'm still in the "Maybe"
state myself. While I'm definitely sympathetic to the proposal
(SSL/TLS is incredibly important, but cross-distro and cross-version
SSL/TLS is already painful, and it's even worse once you bring
cross-platform considerations and the sync/async split into the mix),
it's an unfortunate fact of redistributor life that we don't typically
have the luxury of pitching backport proposals to product management
based on the benefits they offer to the upstream development community
- we need to be able to articulate a clear and relatively immediate
benefit to customers or commercial partners.

For this PEP, "We want to make the new Python TLS abstraction API
work" would potentially qualify as such a benefit (although I can't
make any promises about that), but we're not even at that stage yet -
that's a point we'd only reach if the library was initially built with
a dependency on 3.5+ and 2.7.14+, *and* we either saw significant
demand for that new API amongst customers, or else we genuinely needed
to bring it in as a dependency of something else (and even then, we
prefer approaches like a clearly distinct PyOpenSSL dependency that
can be handled in a higher level product like OpenStack rather than
needing to be baked directly into the operating system itself).

As a result, as awkward and annoying as it would be, I'm wondering if
the structure of the requests migration may need to be something like:

- switch unconditionally to an async backend on Py3
- switch conditionally to an async backend on Py2 when PyOpenSSL is available
- retain sufficient sync-only support on Py2 to allow pip to disable
the PyOpenSSL dependency

As a redistributor, ensuring that sufficiently recent versions of
PyOpenSSL are available is likely to be a *far* more tractable problem
than getting backports implemented at the standard library level (even
as a private API), and even if that doesn't happen, developers that
are using 3rd party modules like requests usually have significantly
more freedom to install additional Python level dependencies than they
do to upgrade their production Python runtime.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia