2.3 -> 2.4: long int too large to convert to int

[Grant Edwards]

On 2005-09-15, Terry Reedy <tjreedy at udel.edu> wrote:

>>I give up, how do I make this not fail under 2.4?
>> 
>> fcntl.ioctl(self.dev.fileno(),0xc0047a80,struct.pack("HBB",0x1c,0x00,0x00))
>>
>> I get an OverflowError: long int too large to convert to int
>>
>> ioctl() is expecting a 32-bit integer value, and 0xc0047a80 has
>> the high-order bit set.  I'm assuming Python thinks it's a
>> signed value.  How do I tell Python that 0xc0047a80 is an
>> unsigned 32-bit value?
>
> In 2.3 and before, you get this:
>>>> 0xc0047a80
> -1073448320

I don't particular care how Python prints the value -- I just
want that value passed to the function I'm calling.

> In 2.4, positive hex literals are treated as positive numbers, and that is 
> your problem: your literal is greater than the largest int and hence gets 
> stored as long int.

I knew that, I just couldn't come up with a good way to fix it.

> I would try -1073448320 as the arg.

That should work, but it's kind of lame (no offense).

ioctl values are always, always written in hex.  A block of
ioctl values is generally assigned to a particular driver such
that the high order N (is it 4 oe 5?) hex digits are unique to
that driver.  Writing the value in decimal is going to
completely confuse anybody looking at the code.

I rather like the other suggestion of writing a function that
accepts 0x<whatever> and returns the appropriate integer value.

Another poster suggested a solution using struct.  Here's my
solution (which assume python integers are represented in 2's
compliment binary):

def ioctlValue(i):
    if i & 0x80000000:
        i = -((i^0xffffffff)+1)
    return i

-- 
Grant Edwards                   grante             Yow!  Somewhere in Tenafly,
                                  at               New Jersey, a chiropractor
                               visi.com            is viewing "Leave it to
                                                   Beaver"!