[Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

[Nathaniel Smith]

On Thu, Jun 9, 2016 at 3:22 PM, Larry Hastings <larry at hastings.org> wrote:
>
> On 06/09/2016 08:52 AM, Guido van Rossum wrote:
>
>> That leaves direct calls to os.urandom(). I don't think this should block
>> either.
>
>
> Then it's you and me against the rest of the world ;-)
>
>
> Okay, it's decided: os.urandom() must be changed for 3.5.2 to never block on
> a getrandom() call.  It's permissible to take advantage of
> getrandom(GRND_NONBLOCK), but if it returns EAGAIN we must read from
> /dev/urandom.
>
> It's already well established that this will upset the cryptography experts.
> As a concession to them, I propose adding a simple! predictable! function to
> Python 3.5.2: os.getrandom().  This would be a simple wrapper over
> getrandom, only available on platforms that expose it.  It would provide a
> way to use both extant flags, GRND_RANDOM and  GRND_NONBLOCK, though
> possibly not exactly mirroring the native API.
>
> This would enable cryptography libraries to easily do what (IIUC) they
> regard as the "correct" thing on Linux for all supported versions of Python:
>
> if hasattr(os, "getrandom"):
>     bits = os.getrandom(n)
> else:
>     bits = os.urandom(n)

So I understand that the trade-offs between crypto users and regular
users are tricky, but this resolution concerns me quite a bit :-(

Specifically, it seems to me that:
1) we now have these two functions that need to be supported forever,
and AFAICT in every case where someone is currently explicitly calling
os.urandom and the behavior differs, they want os.getrandom instead.
(This is based on the assumption that the only time that explicitly
calling os.urandom is the best option is when one cares about the
cryptographic strength of the result -- I'm explicitly distinguishing
here between the hash seeding issue that triggered the original bug
report and explicit calls to os.urandom.) So in practice this change
makes it so that the only correct way of calling either of these
functions is the if/else stanza above.
2) every piece of security-sensitive software is going to spend
resources churning their code to implement the above,
3) every future security audit of Python software is going to spend
resources making sure this is on their checklist of incredibly subtle
gotchas that have to be audited for,
4) the crypto folks are going to have to spin up a whole evangelism
effort to re-educate everyone that (contrary to what we've been
telling everyone for years), os.urandom is no longer the right way to
get cryptographic randomness.

OTOH if we allow explicit calls to os.urandom to block or raise an
exception, then AFAICT from this thread this will break exactly zero
projects.

Maybe this is just rehashing the same things that have already been
discussed ad naseaum, in which case I apologize. But I really feel
like this is one of those cases where the crypto folks aren't so much
saying "oh BUT what if <incredibly unlikely situation involving
oppressive regimes and ticking bombs>"; they're more saying "oh $#@
you're going to cause me a *massive* amount of real work and churn and
ongoing costs for no perceivable gain and I'm exhausted even thinking
about it".

> I'm not excited about adding a new function in 3.5.2, but on the other hand
> we are taking away this functionality they had in 3.5.0 and 3.5.1 so only
> seems fair.  And the implementation of os.getrandom() should be very
> straightforward, and its semantics will mirror the native call, so I'm
> pretty confident we can get it solid in a couple of days, though we might
> slip 3.5.2rc1 by a day or two.
>
> Guido: do you see this as an acceptable compromise?
>
> Cryptographers: given that os.urandom() will no longer block in 3.5.2, do
> you want this?
>
>
> Pointing out an alternate approach: Marc-Andre Lemburg proposes in issue
> #27279 ( http://bugs.python.org/issue27279 ) that we should add two "known
> best-practices" functions to get pseudo-random bits; one merely for pseudo
> random bits, the other for crypto-strength pseudo random bits.  While I
> think this is a fine idea, the exact spelling, semantics, and per-platform
> implementation of these functions is far from settled, and nobody is
> proposing that we do something like that for 3.5.

We already have a function for non-crypto-strength pseudo-random bits:
random.getrandbits. os.urandom is the one for the cryptographers (I
thought).

-n

-- 
Nathaniel J. Smith -- https://vorpus.org