array and struct 64-bit Linux change in behavior Python 3.7 and 2.7

[Chris Angelico]

On Wed, Dec 4, 2019 at 4:16 PM Chris Clark <Chris.Clark at actian.com> wrote:
> I think the consensus from the various threads is that the docs are either lacking or misleading.
>
> I mentioned that this impacts bytes and the problem there is more telling as it hard fails (this is how I first discovered this was an issue):
>
>     >>> array.array('L', b'\0\0\0\0')
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     ValueError: string length not a multiple of item size
>
> I don't believe the documentation is accurate by using the word "minimum". Minimum would suggest that it would accept a 4-byte value as a minimum and on 64-bit it does *not*, it hard fails. If it were to document that, "the sizes are native integer types for the platform, the table documents some typical but *not* guaranteed sizes", that would be more clear.
>

I think array.array() is possibly the wrong tool for this job. If you
have a collection of bytes from some well-defined source (eg you're
parsing a file in a known format), struct is better suited to it,
because it's easy to define both the size and byte order.

> For my uses case I'm seriously thinking about not using array moving forward and only using struct. I briefly wondered about ctypes (it has nice names, e.g. c_int64 that are unambiguous) but then I remembered it is not available in Jython).
>

I wouldn't bother with ctypes for this type of job.

> Regarding Barry's comment, yep size consistency with array is a pain - what I implemented as workaround is below (and likely to be my solution going forward):
>
>     x = array.array('L', [0])
>     if x.itemsize == 4:
>         FMT_ARRAY_4BYTE = 'L'
>         FMT_STRUCT_4BYTE = '<L'
>     else:
>         x = array.array('I', [0])
>         if x.itemsize == 4:
>             FMT_ARRAY_4BYTE = 'I'
>             FMT_STRUCT_4BYTE = '<L'
>     del(x)
>
> and then use the constants in array/struct calls where (binary) file IO is happening.

Yep, looks like struct is the way to go here. (Especially since you
don't have a final 'else'.)

ChrisA