[Python-Dev] Unicode and Windows

[M.-A. Lemburg]

Jack Jansen wrote:
> 
> > "es#",&encoding,&buffer,&buffer_len
> >       -- could mean: coerce the object to Unicode, then
> >          encode it using the given encoding and then
> >          copy at most buffer_len bytes of data into
> >          buffer and update buffer_len to the number of bytes
> >          copied
> 
> This is a possible solution, but I think I would really prefer to also have
>  "eS", &encoding, &buffer_ptr
>  -- coerce the object to Unicode, then encode it using the given
>     encoding, malloc() a buffer to put the result in and return that.
> 
> I don't mind doing something like
> 
> {
>    char *filenamebuffer = NULL;
> 
>    if ( PyArg_ParseTuple(args, "eS", &macencoding, &filenamebuffer)
>        ...
>    open(filenamebuffer, ....);
>    PyMem_XDEL(filenamebuffer);
>    ...
> }
> 
> I think this would be much less error-prone than having fixed-length buffers
> all over the place.

PyArg_ParseTuple() should probably raise an error in case the
data doesn't fit into the buffer.

> And if this is indeed going to be used mainly in open()
> calls and such the cost of the extra malloc()/free() is going to be dwarfed by
> what the underlying OS call is going to use.

Good point. You'll still need the buffer_len output parameter
though -- otherwise you wouldn't be able tell the size of the
allocated buffer (the returned data may not be terminated).

How about this:

"es#", &encoding, &buffer, &buffer_len
	-- both buffer and buffer_len are in/out parameters
	-- if **buffer is non-NULL, copy the data into it
	   (at most buffer_len bytes) and update buffer_len
	   on output; truncation produces an error
	-- if **buffer is NULL, malloc() a buffer of size
	   buffer_len and return it through *buffer; if buffer_len
	   is -1, the allocated buffer should be large enough
	   to hold all data; again, truncation is an error
	-- apply coercion and encoding as described above

(could be that I've got the '*'s wrong, but you get the picture...:)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/