int(long(-sys.maxint-1)) fails on Linux

[Chad Netzer]

Tim Peters wrote:

> Chad, let's break this down to see where the overflow is coming from.  Try
> this line by line and post your results:
>
> >>> import sys
> >>> sys.maxint
> 2147483647
> >>> -sys.maxint
> -2147483647
> >>> -sys.maxint-1
> -2147483648
> >>> long(-sys.maxint-1)
> -2147483648L
> >>> int(long(-sys.maxint-1))
> -2147483648
> >>>
>

>>> import sys
>>> sys.maxint
2147483647
>>> -sys.maxint
-2147483647
>>> -sys.maxint-1
-2147483648
>>> long(-sys.maxint-1)
-2147483648L
>>> int(long(-sys.maxint-1))
Traceback (innermost last):
  File "<stdin>", line 1, in ?
OverflowError: long int too long to convert
>>>

This is on a Pentium III system running straight Mandrake 6.0.  I think, as Christian
and Oliver pointed out, it is pgcc related.  I tried it on a Mandrake 5.3 system (using
the normal gcc, not pgcc, and it worked correctly)

For kicks, I fired up the debugger, and traced through the PyLongASLong call to see
what was happening.  The problem is at the line:

if ((long)x < 0 && (sign > 0 || (x << 1) != 0))

Even though it should evaluate to 0 (and appears to from the debugger's point of view),
it incorrectly proceeds to the "goto overflow" statement.  I looked at the assembly language
output from a working version and a non-working version (both 1.5.2 on Pentiums), and
found that the working version tends to use "cmpl", "testl", etc. and the non-working version
doesn't.  Here is what I mean, the gist of the code is apparent, even though I'm no wizard at
386 asm (I'll provide some comments):

C code:
   if ((long)x < 0 && (sign > 0 || (x << 1) != 0))
            goto overflow;
    return (long)x * sign;


working asm:
<PyLong_AsLong+140>:  cmpl   $0x0,0xfffffffc(%ebp)                      ; if x<0
<PyLong_AsLong+144>:  jnl    0x806adbc <PyLong_AsLong+172>
<PyLong_AsLong+146>:  cmpl   $0x0,0xfffffff0(%ebp)                      ; if sign > 0
<PyLong_AsLong+150>:  jg     0x806adb8 <PyLong_AsLong+168>
<PyLong_AsLong+152>:  movl   0xfffffffc(%ebp),%esi
<PyLong_AsLong+155>:  leal   0x0(,%esi,2),%eax                               ; x<<0 (?)
<PyLong_AsLong+162>:  testl  %eax,%eax
<PyLong_AsLong+164>:  jne    0x806adb8 <PyLong_AsLong+168>
<PyLong_AsLong+166>:  jmp    0x806adbc <PyLong_AsLong+172>
<PyLong_AsLong+168>:  jmp    0x806adc8 <PyLong_AsLong+184>  ; goto overflow
<PyLong_AsLong+170>:  movl   %esi,%esi                                           ; return (long)x * sign;
<PyLong_AsLong+172>:  movl   0xfffffffc(%ebp),%edx
<PyLong_AsLong+175>:  imull  0xfffffff0(%ebp),%edx
<PyLong_AsLong+179>:  movl   %edx,%eax
<PyLong_AsLong+181>:  jmp    0x806ade4 <PyLong_AsLong+212>

not working:
<PyLong_AsLong+150>: cmpl   $0x0,0xfffffffc(%ebp)
<PyLong_AsLong+154>: jge    0x806c3f2 <PyLong_AsLong+178>
<PyLong_AsLong+156>: cmpl   $0x0,0xfffffff0(%ebp)
<PyLong_AsLong+160>: jg     0x806c3f0 <PyLong_AsLong+176>
<PyLong_AsLong+162>: mov    0xfffffffc(%ebp),%eax
<PyLong_AsLong+165>: add    %eax,%eax                                          ;  x<<1
<PyLong_AsLong+167>: test   %eax,%eax
<PyLong_AsLong+169>: jne    0x806c3f0 <PyLong_AsLong+176>
<PyLong_AsLong+171>: jmp    0x806c3f2 <PyLong_AsLong+178>
<PyLong_AsLong+173>: lea    0x0(%esi),%esi
<PyLong_AsLong+176>: jmp    0x806c400 <PyLong_AsLong+192> ; goto overflow
<PyLong_AsLong+178>: mov    0xfffffffc(%ebp),%edx                     ; return (long)x * sign;
<PyLong_AsLong+181>: imul   0xfffffff0(%ebp),%edx
<PyLong_AsLong+185>: mov    %edx,%eax
<PyLong_AsLong+187>: jmp    0x806c41a <PyLong_AsLong+218>

If nyone who knows more about assembly, I would mind hearing where the problem
is, and to know if it is worth submitting a bug-report.

It sounds as if Oliver's very-soon-to-be-released versions of the packages may have
fixed the problem w/ compiler flags.  I'm anxious to try it out (Oliver, thanks for all
the effort, BTW.  I got impatient waiting for your new releases, and have been making
my own based on you specs and packages...  LOT of work getting everything just right!
I figured you were busy, as we all are, but I'm happy to see that you'll be releasing soon.
I will definately join the python4linux mailing list, as I would like to help out in the
future)

Thanks for the help everyone.

Chad Netzer
chad at vision.arc.nasa.gov