Writev

[Steven Bethard]

Adam DePrince wrote:
> file.writelines( seq ) and map( file.write, seq ) are the same; the
> former is syntactic sugar for the later.

Well, that's not exactly true.  For one thing, map(file.write, seq) 
returns a list of Nones, while file.writelines returns only the single 
None that Python functions with no return statement do.  More 
substantially, file.writelines (as far as I can tell from the C code) 
doesn't make any call to file.write.

I looked at fileobject.c and it looks like file.writelines makes a call 
to 'fwrite' for each item in the iterable given.  Your code, if I read 
it right, makes a call to 'writev' for each item in the iterable.

I looked at the fwrite() and writev() docs and read your comments, but I 
still couldn't quite figure out what makes 'writev' more efficient than 
'fwrite' for the same size buffer...  Mind trying to explain it to me again?

> There is one more time that writev would be beneficial ... perhaps you
> want to write a never ending sequence with a minimum of overhead? 
> 
> def camera():
> 	while 1:
> 		yield extract_entropy( grab_frame() )
> 
> open( "/tmp/entropy_daemon_pipe", "w+" ).writev( camera(), 5 ) 

I tried running:

py> def gen():
...     i = 1
...     while True:
...         yield '%i\n' % i
...         i *= 10
...
py> open('integers.txt', 'w').writelines(gen())

and, while (of course) it runs forever, I don't appear to get any memory 
problems.  I seem to be able to write fairly large integers too:

$ tail -n 1 integers.txt | wc
       1       1    5001

How big do the items in the iterable need to be for writev to be necessary?

Steve

P.S.  I certainly don't have anything against including your patch (not 
that my opinion counts for anything) ;) but if it improves a common 
file.writelines usage, I'd like to see it used there too when possible.