array and struct 64-bit Linux change in behavior Python 3.7 and 2.7

[Christian Heimes]

On 02/12/2019 22.25, Barry Scott wrote:
> 
> 
>> On 2 Dec 2019, at 17:55, Rob Gaddi <rgaddi at highlandtechnology.invalid> wrote:
>>
>> On 12/2/19 9:26 AM, Chris Clark wrote:
>>> Test case:
>>>                import array
>>>                array.array('L', [0])
>>> # x.itemsize == 8  rather than 4
>>> This works fine (returns 4) under Windows Python 3.7.3 64-bit build.
>>> Under Ubuntu; Python 2.7.15rc1, 3.6.5, 3.70b3 64-bit this returns 8. Documentation at https://docs.python.org/3/library/array.html explicitly states 'L' is for size 4.
>>> It impacts all uses types of array (e.g. reading from byte strings).
>>> The struct module is a little different:
>>> import struct
>>> x = struct.pack('L', 0)
>>> # len(x) ===8 rather than 4
>>> This can be worked around by using '=L' - which is not well documented - so this maybe a doc issue.
>>> Wanted to post here for comments before opening a bug at https://bugs.python.org/
>>> Is anyone seeing this under Debian/Ubuntu?
>>> Chris
>>
>> I'd say not a bug, at least in array.  Reading that array documentation you linked, 4 is explicitly the MINIMUM size in bytes, not the guaranteed size.
> 
> I'm wondering how useful it is that for array you can read from a file but have no ideas how many bytes each item needs.
> If I have a file with int32_t  in it I cannot from the docs know how to read that file into an array.
> 
>>
>> The struct situation is, as you said, a bit different.  I believe that with the default native alignment @, you're seeing 4-byte data padded to an 8-byte alignment, not 8-byte data.  That does seem to go against what the struct documentation says, "Padding is only automatically added between successive structure members. No padding is added at the beginning or the end of the encoded struct."
> 
> The 'L' in struct is documented for 3.7 to use 4 bytes, but in fact uses 8, on fedora 31. Doc bug?

The documentation of the struct and array module carefully speak of
"minimum size in bytes", "standard size" and "native size". It's easy to
miss that it doesn't state just "size" for a reason.

A long is not int32_t. The actual size of long and unsigned long depend
on the ABI of your platform. The standard defined a long as *at least* 4
bytes. On Windows it's always a 32 bit data type. On POSIX sizeof(long)
is usually the same as the size of a pointer, 4 bytes on 32 bit
platforms and 8 bytes on 64 bit platforms.

https://en.wikipedia.org/wiki/Integer_(computer_science)#Long_integer

I agree that the behavior is confusing, even for C programmers. Please
feel free to open a ticket and request an improvement of the documentation.

Christian