performance of script to write very long lines of random chars

[gry]

On Apr 10, 9:52 pm, Michael Torrie <torr... at gmail.com> wrote:
> On 04/10/2013 07:21 PM, gry wrote:
>
>
>
>
>
>
>
>
>
> > from sys import stdout
> > from array import array
> > import random
> > nchars = 32000000
> > rows = 10
> > avail_chrs =
> > '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&
> > \'()*+,-./:;<=>?@[\\]^_`{}'
> > a = array('c', 'X' * nchars)
>
> > for l in range(rows):
> >     for i in xrange(nchars):
> >         a[i] = random.choice(avail_chrs)
> >     a.tofile(stdout)
> >     stdout.write('\n')
>
> > This version using array took 4 min, 29 sec, using 34MB resident/110
> > virtual. So, much smaller than the first attempt, but a bit slower.
> > Can someone suggest a better code?  And help me understand the
> > performance issues here?
>
> Why are you using an array?  Why not just rely on the OS to buffer the
> output.  Just write your characters straight to stdout instead of
> placing them in an array.
>
> At that point I believe this program will be as fast as is possible in
> Python.

Appealing idea, but it's slower than the array solution: 5min 13
secs.  vs 4min 30sec for the array:

for l in range(rows):
    for i in xrange(nchars):
        stdout.write(random.choice(avail_chrs))
    stdout.write('\n')


os.urandom does look promising -- I have to have full control over the
charset, but urandom is very fast at generating big random strings...
stay tuned...