Make a unique filesystem path, without creating the file

[Steven D'Aprano]

On Wednesday 24 February 2016 18:20, Marko Rauhamaa wrote:

> Steven D'Aprano <steve at pearwood.info>:
> 
>> On Tue, 23 Feb 2016 05:54 pm, Marko Rauhamaa wrote:
>>> However, when you are generating signing or encryption keys, you
>>> should use /dev/random.
>>
>> And that is where you repeat something which is rank superstition.
> 
> Can you find info to back that up. 

The links already provided go through the evidence. For example, they 
explain that /dev/random and /dev/urandom both use the exact same CSPRNG. If 
you don't believe that, you can actually read the source to Linux, FreeBSD, 
OpenBSD and NetBSD. (But not OS X, sorry.)

Put aside the known bug in Linux, where urandom will provide predictable 
values in the period following a fresh install before the OS has collected 
enough entropy. We all agree that's a problem, and that the right solution 
is to block. The question is, outside of that narrow set of circumstances, 
when it is appropriate to block?

As I mentioned, most Unixes don't block. urandom and random behave exactly 
the same way in three of the most popular Unixes (FreeBSD, OpenBSD, OS X):

https://en.wikipedia.org/wiki//dev/random

so let's consider just those that do block (Linux, and NetBSD). They both 
use the same CSPRNG. Do you dispute that? Then read the source. For one to 
be "better" than the other, there would need to be a detectable difference 
between the two. Nobody has ever found one, and nor will they, because 
they're both coming from the same CSPRNG (AES in the case of NetBSD, I'm not 
sure what in the case of Linux).

If you don't trust the CSPRNG, then you shouldn't trust it whether it comes 
from /dev/random or /dev/urandom. If you do trust it, then why would you 
want it to block? Blocking doesn't make it more random. That's not how it 
works. It just makes you vulnerable to a Denial Of Service attack.

There really doesn't seem to be any valid reason for random blocking. It's 
like warnings and timers on fans in South Korea to prevent fan death:

http://www.snopes.com/medical/freakish/fandeath.asp

(My favourite explanation is that the blades of the fan chop the oxygen 
molecules in two.)

I'm not surprised that there is so much misinformation about random/urandom. 
Here's a blog post by somebody wanting to port arc4 to Linux, so he clearly 
knows a few things about crypto. I can't judge whether arc4 is better or 
worse than what Linux already uses, but look at this quote:

http://insanecoding.blogspot.com.au/2014/05/a-good-idea-with-bad-usage-
devurandom.html

Quote:

    Linux is well known for inventing and supplying two default files,
    /dev/random and /dev/urandom (unlimited random). The former is 
    pretty much raw entropy, while the latter is the output of a CSPRNG
    function like OpenBSD's arc4random family. The former can be seen 
    as more random, and the latter as less random, but the differences
    are extremely hard to measure, which is why CSPRNGs work in the 
    first place. Since the former is only entropy, it is limited as to
    how much it can output, and one needing a lot of random data can be
    stuck waiting a while for it to fill up the random buffer. Since the
    latter is a CSPRNG, it can keep outputting data indefinitely, 
    without any significant waiting periods."

There's that myth about urandom being "less random" than random again, but 
even this guy admits that the difference is "extremely hard" (actually: 
impossible) to measure, and that CSPRNG's "work". Which is precisely why 
OpenBSD uses arc4random for their /dev/random and /dev/urandom, and 
presumably why he wants to bring it to Linux.

This author is *completely wrong* to say that /dev/random is "pretty much 
raw entropy". If it were, it would be biased, and easily manipulated by an 
attacker. Entropy is collected from (among other things) network traffic, 
which would allow an attacker to control at least one source of entropy and 
hence (in theory) make it easier to predict the output of /dev/random.

But fortunately it is not true. Linux's random system works like this:

- entropy is collected from various sources and fed into a pool;

- entropy from that pool is fed through a CSPRNG into two separate pools, 
  one each for /dev/random and /dev/urandom;

- when you read from /dev/random or urandom, they both collect entropy
  from their own pool, and again pass it through a CSPRNG;

- /dev/random has a throttle (it blocks if you take out too much);

- /dev/urandom doesn't have a throttle.

https://events.linuxfoundation.org/images/stories/pdf/lceu2012_anvin.pdf


Somebody criticized the author for spreading this misapprehension that 
/dev/random is "raw entropy" and here is his response:

    I tried giving an explanation which should be simple for a 
    layman to follow of what goes on. I wouldn't take it as precise 
    fact, especially when there's a washing machine involved in the
    explanation ;)


Or, in other words, "When I said the moon was made of green cheese, I was 
simplifying it for the layman. I didn't mean it was precisely green cheese. 
It's just like green cheese."

Sure. But it is *more like* rock and dust than green cheese. In fact, apart 
from both being made of atoms, it's hard to see any similarity between the 
moon and green cheese at all, just as there is no real similarity between 
what /dev/random actually does, and what this guy says it does.

The bottom line is, nobody can distinguish the output of urandom and random 
(apart from the blocking behaviour). Nobody has demonstrated any way to 
distinguish the output of either random or urandom from "actual randomness". 
There are theoretical attacks on urandom that random might be immune to, but 
if so, I haven't heard what they are.

Apart from pointing you at the source code, I don't know what else I can say 
to prove this. I can't prove that there's no known attack on Linux's CSPRNG, 
just as I can't prove that there's no tea cup in orbit around Jupiter, or 
that there wasn't a nuclear war between China and the USSR in 1985 and both 
countries covered it up. Maybe there was. But I wouldn't bet on it.


> All I've seen so far is forceful
> claims that's superstition ("These are not the droids you're looking
> for"). 
>
> Even the ssh-keygen man page has:
> 
>     The reseeding of the OpenSSL random generator is usually done from
>     /dev/urandom. If the SSH_USE_STRONG_RNG environment vari‐ able is
>     set to value other than 0 the OpenSSL random generator is reseeded
>     from /dev/random.

If they had called the environment variable SSH_MAKE_MONEY_FAST would you 
believe it?

What evidence do they give that /dev/urandom is weak? If it is weak, why are 
they using it as the default?



-- 
Steve