[Python-checkins] fix up signedness in PyImport_ExtendInittab (#4831)

[Benjamin Peterson]

https://github.com/python/cpython/commit/0c644fcda095b12eedfb417f15429795c09196b6
commit: 0c644fcda095b12eedfb417f15429795c09196b6
branch: master
author: Benjamin Peterson <benjamin at python.org>
committer: GitHub <noreply at github.com>
date: 2017-12-15T23:42:33-08:00
summary:

fix up signedness in PyImport_ExtendInittab (#4831)

As a result of 92a3c6f493ad411e4cf0acdf305ef4876aa90669, the compiler complains:

Python/import.c:2311:21: warning: comparison of integers of different signs: 'long' and 'unsigned long' [-Wsign-compare]
    if ((i + n + 1) <= PY_SSIZE_T_MAX / sizeof(struct _inittab)) {
         ~~~~~~~~~  ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This overflow is extremely unlikely to happen, but let's avoid undefined
behavior anyway.

files:
M Python/import.c

diff --git a/Python/import.c b/Python/import.c
index d5dad1a6a30..8d306403a29 100644
--- a/Python/import.c
+++ b/Python/import.c
@@ -2291,7 +2291,7 @@ int
 PyImport_ExtendInittab(struct _inittab *newtab)
 {
     struct _inittab *p;
-    Py_ssize_t i, n;
+    size_t i, n;
     int res = 0;
 
     /* Count the number of entries in both tables */
@@ -2308,13 +2308,11 @@ PyImport_ExtendInittab(struct _inittab *newtab)
     _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
 
     /* Allocate new memory for the combined table */
-    if ((i + n + 1) <= PY_SSIZE_T_MAX / (Py_ssize_t)sizeof(struct _inittab)) {
+    p = NULL;
+    if (i + n <= SIZE_MAX / sizeof(struct _inittab) - 1) {
         size_t size = sizeof(struct _inittab) * (i + n + 1);
         p = PyMem_RawRealloc(inittab_copy, size);
     }
-    else {
-        p = NULL;
-    }
     if (p == NULL) {
         res = -1;
         goto done;