Does shuffle() produce uniform result ?

[Alex Martelli]

tooru honda <tooru_honda at fast-mail.org> wrote:

> At the end, I think it is worthwhile to implement my own shuffle and 
> random methods based on os.urandom.  Not only does the resulting code
> gets rid of the minuscule bias, but the program also runs much faster.
> 
> When using random.SystemRandom.shuffle, posix.open and posix.close from
> calling os.urandom account for almost half of the total execution time
> for my program.  By implementing my own random and getting a much larger
> chunk of random bytes from os.urandom each time, I am able to reduce the
> total execution time by half.

If I were in your shoes, I would optimize by subclassing
random.SystemRandom and overriding the random method to use os.urandom
with some large block size and then parcel it out, instead of the
_urandom(7) that it now uses.  E.g., something like:

class SystemBlockRandom(random.SystemRandom):

    def __init__(self):
        random.SystemRandom.__init__(self)
        def rand7():
            while True:
                randata = os.urandom(7*1024)
                for i in xrange(0, 7*1024, 7):
                    yield long(binascii.hexlify(randata[i:i+7]),16)
        self.rand7 = rand7().next

    def random(self):
        """Get the next random number in the range [0.0, 1.0)."""
        return (self.rand7() >> 3) * random.RECIP_BPF

(untested code).  No need to reimplement anything else, it seems to me.


Alex