[issue1621] Do not assume signed integer overflow behavior

[Mark Dickinson]

Mark Dickinson <dickinsm at gmail.com> added the comment:

Thanks Ismail.  I'm currently using gcc-4.4 with the -ftrapv (not 
-fwrapv!) option to see how much breaks.  (Answer: quite a lot. :-[ )

I'm finding many overflow checks that look like:

	size = Py_SIZE(a) * n;
	if (n && size / n != Py_SIZE(a)) {
		PyErr_SetString(PyExc_OverflowError,
			"repeated bytes are too long");
		return NULL;
	}

where size and n have type Py_ssize_t.  That particular one comes
from bytesobject.c (in py3k), but this style of check occurs
frequently throughout the source.

Do people think that all these should be fixed?  

The fix itself s reasonably straightforward:  instead of multiplying
and then checking for an overflow that's already happened (and hence
has already invoked undefined behaviour according to the standards),
get an upper bound for n *first* by dividing PY_SSIZE_T_MAX
by Py_SIZE(a) and use that to do the overflow check *before*
the multiplication.  It shouldn't be less efficient:  either way
involves an integer division, a comparison, and a multiplication.

The hard part is finding all the places that need to be fixed.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1621>
_______________________________________