[Python-Dev] Crazy idea for str.join

[Josiah Carlson]

Edward Loper <edloper at gradient.cis.upenn.edu> wrote:
> 
> Nick Coghlan wrote:
> > Edward Loper wrote:
> >> This is incompatible with the recent proposal making str.join 
> >> automatically str-ify its arguments.  i.e.:
> >>
> >>    ''.join(['a', 12, 'b']) -> 'a12b'.
> >>
> >> I don't feel strongly about either proposal, I just thought I'd point 
> >> out that they're mutually exclusive.
> > 
> > Doesn't accepting objects that support the buffer interface come for 
> > free with stringification? (My understanding of buffer objects is fairly 
> > sketchy, so I may be missing something here. . .)
> 
> It's quite possible that I'm the one who was confused.. I just tried it 
> out with string-based buffers, and for that case at least it works fine. 
>   I don't know about other buffers.  So actually these two proposals 
> might be compatible after all.

The point of my proposal was to prevent an unnecessary level of copying
during a str.join .  Right now, if one manually uses str(buf) (or
indexing or slicing), it copies the contents of the buffer into a string
(likely via PyString_FromStringAndSize() ).  By using
PyObject_AsCharBuffer(), and using memcpy on that pointer rather than
the pointer returned by PyString_AsString() from any (possibly
auto-stringified) strings, it would reduce memory copies by half for
those non-string buffer-supporting arguments.

At least for the examples of buffers that I've seen, using the buffer
interface for objects that support it is equivalent to automatically
applying str() to them.  This is, strictly speaking, an optimization.

 - Josiah