[Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

[Christian Heimes]

On 2016-06-09 13:25, Larry Hastings wrote:
> 
> A problem has surfaced just this week in 3.5.1.  Obviously this is a
> good time to fix it for 3.5.2.  But there's a big argument over what is
> "broken" and what is an appropriate "fix".
> 
> As 3.5 Release Manager, I can put my foot down and make rulings, and
> AFAIK the only way to overrule me is with the BDFL.  In two of three
> cases I've put my foot down.  In the third I'm pretty sure I'm right,
> but IIUC literally everyone with a stated opinion else disagrees with
> me.  So I thought it best I escalate it.  Note that 3.5.2 is going to
> wait until the issue is settled and any changes to behavior are written
> and checked in.
> 
> (Blanket disclaimer for the below: in some places I'm trying to
> communicate other's people positions.  I apologize if I misrepresented
> yours; please reply and correct my mistake.  Also, sorry for the length
> of this email.  But feel even sorrier for me: this debate has already
> eaten two days this week.)

Thanks for the digest, Larry.

I would appreciate if we could split the issue into three separate problems:

1) behavior of os.urandom()
2) initialization of _Py_HashSecret for byte, str and XML hash
randomization.
3) initialization of default random.random Mersenne-Twister

As of now 2 and 3 are the culprit for blocking starting. Both happen to
use _PyOS_URandom() either directly or indirectly through os.urandom().
We chose to use the OS random source because it was convenient. It is
not a necessity. The seed for Mersenne-Twister and the keys for hash
randomization don't have to be strong cryptographic values in all cases.
They just have to be hard-to-guess by an attacker. In case of scripts in
early boot, there are no viable attack scenarios.

Therefore I propose to fix problem 2 and 3:

- add a new random_seed member to _Py_HashSecret and use it to derive an
initial Mersenne-Twister state for the default random instance of the
random module.

- try CPRNG for _Py_HashSecret first, fall back to a user space RNG when
the Kernel's CPRNG would block.

For some operating systems like Windows and OSX, we can assume that
Kernel CPRNG is always available. For Linux we can use getrandom() in
non-blocking mode and handle EWOULDBLOCK. On BSD the seed state can be
queried from /proc.

Christian