[Security-sig] Take a decision for os.urandom() in Python 3.6

[Nick Coghlan]

On 6 August 2016 at 20:23, Victor Stinner <victor.stinner at gmail.com> wrote:
> An alternative would be to add to my PEP 524 an *optional*
> random.SystemRandomNonblock which is basically the PEP 522 (raise if it
> would block). "Optional"... or maybe make it always available but block(!)
> on some platforms? (Bad idea IMO)

No, we don't want anything new added to the random module for this -
outside backwards compatibility considerations, random.SystemRandom
should be superseded entirely for security sensitive purposes by the
module level APIs in the secrets module.

> I dislike the idea of adding 2 new functions to generate random in the same
> PEP (getrandom, SystemRandomNonblock), it's already hard enough to pick to
> right one in Python 3.5...

With the changes to PEP 522, secrets.token_bytes will be a
cross-platform blocking API regardless of which underlying
implementation model we choose - either inheriting that behaviour from
os.urandom() (PEP 524), or adding it when encountering BlockingIOError
(PEP 522).

That means the essential question becomes: Should os.urandom() just be
secrets.token_bytes() without a default number of bytes requested? Or
does it make more sense to use it to expose the Linux sys.getrandom()
non-blocking behaviour to Python code in a platform independent way?

Since we're going to have the two level API anyway (os module vs
secrets), and have two different behaviours we'd like to expose
(blocking vs non-blocking with notification), the latter design is the
one I ended up converging on: high level API with implicit blocking,
low level API that never blocks, but may throw an exception. It's not
where I expected to end up when I first wrote the PEP, but that's the
PEP process for you :)

Cheers,
Nick.

>
> Victor
>
>
> Le 6 août 2016 10:46 AM, "Nick Coghlan" <ncoghlan at gmail.com> a écrit :
>>
>> On 6 August 2016 at 18:32, Victor Stinner <victor.stinner at gmail.com>
>> wrote:
>> > Le 6 août 2016 04:39, "Guido van Rossum" <guido at python.org> a écrit :
>> >> 4. Bad code gets cargo-culted (e.g. through StackOverflow).
>> >
>> >> Re (4): With PEP 524, people worried about blocking may be driven to
>> >> unnecessarily write more complicated code using os.getrandom(). With
>> >> PEP 522, people worried about crashes may be driven to unnecessarily
>> >> call secrets.wait_for_system_rng() or put try/except blocks catching
>> >> raise BlockingIOError around all their os.urandom()-based calls.
>> >
>> > What can we do to reduce this issue? Promote the best recipes in the
>> > documentation of the random and/or secrets module? Add Nick's
>> > secrets.wait_for_system_rng()?
>>
>> At the moment, PEP 522 doesn't propose making the secrets API block
>> implicitly. I was already starting to have doubts about that, and
>> given Guido's feedback, I think I should change it so that it does.
>>
>> That would give the following overall outcome:
>>
>> - the random APIs will never block (but shouldn't be used for secrets)
>> - the secrets APIs will block if they need to (including
>> secrets.wait_for_system_rng())
>> - os.urandom() may raise BlockingIOError if you don't wait for the
>> system RNG first
>> - random.SystemRandom() may raise BlockingIOError if you don't wait
>> for the system RNG first
>>
>> And if in the latter two cases someone is directed to the secrets
>> module to wait for the system RNG to be ready (e.g. in the error
>> message we raise), they may find that secrets offers a higher level
>> API for whatever they were trying to do anyway.
>>
>> Meanwhile, folks that want to do something other than block if the
>> system RNG isn't ready (like log potentially relevant details of the
>> system encountering the lack of entropy) can just catch
>> BlockingIOError, rather than needing to use platform specific APIs
>> like os.getrandom().
>>
>> Cheers,
>> Nick.
>>
>> --
>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia