Writev

[Jp Calderone]

On Sun, 19 Dec 2004 23:12:27 -0500, Adam DePrince <adam at cognitcorp.com> wrote:
> [snip]
> 
> Of course, to take advantage of this requires that writev be exposed.  I
> have an implementation of writev.  This implementation is reasonably
> smart, it "unrolls" only so as many iteration.next calls as necessary to
> fill the pointer list for the next invocation of writev.   Of course, on
> many systems (Linux, BSD) writev's limit is 1024 items, so to
> accommodate users who don't want to peel off and hold in memory 1024
> iteration.next()'s, you can set an optional parameter to a smaller
> value. 
> 
> I'm not sure where to take this as a "next step."  It seems too small a
> change for a PEP.  Any ideas?
> 
> You can download the patch from
> http://deprince.net/software/writev/index.html
> 

  writev() in the standard library would be nice.  I have an 
implementation too ;)  I wonder how many other people do as 
well.

  Regarding the implementation, just one part confuses me.  
What's the idea behind this check at the beginning of the 
iterator unrolling loop?

+    if ( !PyObject_CheckReadBuffer( buf ) )
+      continue;

  Is it me, or is that silently ignoring bad input?

  Other miscellaneous feedback:

    The use of alloca() isn't common in CPython.  PyMem_Malloc 
is probably what you want to be calling.  At the very least, 
alloca()'s return value should be checked to be sure it is 
non-NULL.  If you switch to PyMem_Malloc, you'll have to be 
sure to free the memory, of course.

    The support of iterators is a cool idea, but I'm not sure
it is actually useful.  Consider the case where not all bytes 
are written.  How is an application supposed to handle writing 
the rest of the bytes, if they have been consumed from an 
iterator?  Hopefully I'm missing the obvious answer, because
supporting arbitrary iterators would be cool, but it seems like
only sequences can really be usefully supported.

    I agree it's a bit small for a PEP.  I think a patch,
including documentation and unit tests (something along the 
lines of http://cvs.twistedmatrix.com/cvs/*checkout*/trunk/sandbox/exarkun/test_iovec.py?content-type=text%2Fplain&rev=9505&root=Twisted
for example) on sf.net is probably a good next step.  I imagine 
there will be some resistence to adding this for 2.4.1, though,
meaning it may have to wait for 2.5.

  Jp