[New-bugs-announce] [issue9579] In 3.x, os.confstr() returns garbage if value is longer than 255 bytes

[David Watson]

New submission from David Watson <baikie at users.sourceforge.net>:

It may be hard to find a configuration string this long, but you
can see the problem if you apply the attached
confstr-reduce-bufsize.diff to reduce the size of the local array
buffer that posix_confstr() uses.  With it applied:

>>> import os
>>> print(ascii(os.confstr("CS_PATH")))
'\x00\ucbcb\ucbcb\ucbcb\ucbcb\ucbcb\ucbcb\ucbcb\ucbcb\ucbcb\ucbcb\ucbcb\ucbcb'

The problem arises because the code first tries to receive the
configuration string into the local buffer (char buffer[256],
reduced to char buffer[1] above), but then tries to receive it
directly into a string object if it doesn't fit.  You can see
what's gone wrong by comparing the working code in 2.x:

    if ((unsigned int)len >= sizeof(buffer)) {
        result = PyString_FromStringAndSize(NULL, len-1);
        if (result != NULL)
            confstr(name, PyString_AS_STRING(result), len);
    }
    else
        result = PyString_FromStringAndSize(buffer, len-1);

with the code in 3.x:

    if ((unsigned int)len >= sizeof(buffer)) {
        result = PyUnicode_FromStringAndSize(NULL, len-1);
        if (result != NULL)
            confstr(name, _PyUnicode_AsString(result), len);
    }
    else
        result = PyUnicode_FromStringAndSize(buffer, len-1);

Namely, that in 3.x it tries to receive the string into the bytes
object returned by _PyUnicode_AsString(), not the str object it
has just allocated (which has the wrong format anyway -
Py_UNICODE as opposed to char).

The attached confstr-long-result.diff fixes this by allocating a
separate buffer when necessary to receive the result, before
creating the string object from it.  By putting the confstr()
call and allocation in a loop, it also handles the possibility
that the value's length might change between calls.

----------
components: Extension Modules
files: confstr-reduce-bufsize.diff
keywords: patch
messages: 113699
nosy: baikie
priority: normal
severity: normal
status: open
title: In 3.x, os.confstr() returns garbage if value is longer than 255 bytes
type: behavior
versions: Python 3.1, Python 3.2
Added file: http://bugs.python.org/file18486/confstr-reduce-bufsize.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9579>
_______________________________________