[issue26839] Python 3.5 running on Linux kernel 3.17+ can block at startup or on importing the random module on getrandom()

[Theodore Tso]

Theodore Tso added the comment:

I ran the experiment Colm asked me to run --- and yes, if you boot a system with Python 3.5.1 with the boot options "init=/usr/bin/python3", you're going to have a bad time.   The problem is that in a KVM environment where things are very quiet, especially if you are using a tickless kernel, if python calls getrandom(2), it will block since the entropy pool hasn't been initialized yet.   But since we aren't doing anything, the system becomes completely quiescent and so no entropy can be collected.  If systemd tries to run a python script very early in the boot process, and does this in a way where no further boot time activity takes place until the python script exits, you can indeed deadlock the system.

The solution is I think what Donald suggested in msg267746, which is to use GRND_NONBLOCK for initializing the hash which gets used for the dict, or whatever it's used for.   My understanding is that this is not a long-term cryptographic secret, and indeed it will be thrown away as soon as the python interpreter exits.  Since this is before networking has been brought up, the denial service attack or whatever requires that you use a strong SipHash for your Python dictionaries shouldn't be a problem.   (Which I gather has something to do with this?   https://events.ccc.de/congress/2011/Fahrplan/attachments/2007_28C3_Effective_DoS_on_web_application_platforms.pdf)

Now, I can see people being concerned that if Python *always* initializes its hash dictionaries using getrandom with GRND_NONBLOCK, it might be opening up a DOS attack.   Well, in practice, once the boot sequence continues and the system is actually doing some real work, within a few seconds the random number generator will be initialized so in practice it won't be an issue once the system has booted.

If you want to be really paranoid, I suppose you could give some kind of command-line flag which tells Python to use GRND_NONBLOCK for the purposes of initializing its hash table for its dictionary, and only use it in the boot path.   In practice, I suspect very early in the systemd boot path, before it actually starts running the boot scripts in parallel, is the only place where you are likely going to run into this problem, so making it be a flag that only systemd scripts have to call is probably the right thing to do.   But I'll let someone else have the joys of negotiating with Lennart, and I won't blame the Python devs if using GRND_NONBLOCK unconditionally is less painful than having to work with the systemd folks.  :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26839>
_______________________________________