[issue23985] Crash when deleting slices from duplicated bytearray

[Martin Panter]

Martin Panter added the comment:

After cleaning my build and rebuilding with “./configure --with-pymalloc --with-pydebug”, I reduced my script to these four lines:

b1 = bytearray(b"abcdefghij")  # 10 bytes
del b1[:1]
del b1[:1]
b1 += b"klmnopq"  # 7 bytes

Patch bytearray-fix.patch fixes the bug by taking account fact that ob_start is offset into the allocated memory buffer when checking if a reallocation is necessary.

Explanation with the unpatched code and my four-line script above:

1. First line allocates 11 bytes of memory (10 for the byte string + 1 NUL terminator)

2. First “del” reduces the bytearray length to 9 bytes, but actually reallocates an expanded memory buffer of 16 bytes! (quirky but not the bug)

3. Second “del” reduces the bytearray length to 8 bytes, and increments an internal ob_start offset without any memory copying or reallocation. (Fine.)

4. Appending step needs to add 7 bytes to the 8-byte array. 7 + 8 + 1 is 16 bytes total required for bytearray and NUL terminator, but since ob_start is offset from the start of the allocated memory buffer, we overwrite past the end of the buffer.

Memory debugging output and extra debug printfs of my own:

Resizing to size 10 (current log. offset 0 alloc 0)
=> Major upsize, new alloc = 11
Assigning to linear slice
- Shifting ob_start for negative growth -1
Resizing to size 9 (current log. offset 1 alloc 11)
=> Moderate upsize, new alloc = 16
- Done assigning to linear slice
Assigning to linear slice
- Shifting ob_start for negative growth -1
Resizing to size 8 (current log. offset 1 alloc 16)
=> Minor downsize
- Done assigning to linear slice
Debug memory block at address p=0x7f1af630a0d0: API 'o'
    16 bytes originally requested
    The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
    The 8 pad bytes at tail=0x7f1af630a0e0 are not all FORBIDDENBYTE (0xfb):
        at tail+0: 0x00 *** OUCH
        at tail+1: 0xfb
        at tail+2: 0xfb
        at tail+3: 0xfb
        at tail+4: 0xfb
        at tail+5: 0xfb
        at tail+6: 0xfb
        at tail+7: 0xfb
    The block was made by call #32897 to debug malloc/realloc.
    Data at p: 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71
Fatal Python error: bad trailing pad byte

----------
keywords: +patch
Added file: http://bugs.python.org/file39098/bytearray-fix.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23985>
_______________________________________