[issue8674] audioop: incorrect integer overflow checks

[Tomas Hoger]

Tomas Hoger <thoger at redhat.com> added the comment:

> Do you have any Python examples that failed to trigger the overflow
> on your platform?

No, I've not really tried to create some, as I found it while looking into similar checks added to rgbimg module (which is dead and removed upstream now) in the same commit r64114.

Having another close look, I can reproduce crash with lin2lin:
  audioop.lin2lin("A"*0x40000001, 1, 4)

ratecv may cause issues too.  Other cases use for loop with multiplication product as an upper bound, so the integer overflow should be harmless in those case.

> is there something about the formats that audioop is dealing
> with that limits sizes to INT_MAX (rather than PY_SSIZE_T_MAX,
> for example)?

I've started looking into this on oldish python 2.4, where PyString_FromStringAndSize accepts int size, rather than Py_ssize_t.  Rest of the audioop code was using ints too.  It's possible it is ok to more to size_t in current python version.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue8674>
_______________________________________