[Python-Dev] ctypes: Configurable bitfield allocation strategy

[Vlad Riscutia]

This is the cause of both bug reports and yes, it should improve interop
with GCC-compiled code on Windows. My point is that this is not a platform
thing, it's more of a compiler thing. Currently issues appear on Windows for
interop between MSVC Python and other GCC code but since bitfield allocation
is up to the compiler, it could appear on any other platform due to
different compilers being used.

On Sat, Jun 25, 2011 at 12:09 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 6/25/2011 12:33 PM, Vlad Riscutia wrote:
>
>> I recently started looking at some ctypes issues. I dug a bit into
>> http://bugs.python.org/**issue6069 <http://bugs.python.org/issue6069> and
>> then I found
>> http://bugs.python.org/**issue11920 <http://bugs.python.org/issue11920>.
>> They both boil down to the fact that
>> bitfield allocation is up to the compiler, which is different in GCC and
>> MSVC. Currently we have hard-coded allocation strategy based on paltform
>> in cfields.c:
>>
>>   if (bitsize /* this is a bitfield request */
>>>
>>
>>         &&  *pfield_size /* we have a bitfield open */
>>>  #ifdef MS_WIN32
>>>        /* MSVC, GCC with -mms-bitfields */
>>>        &&  dict->size * 8 == *pfield_size
>>>  #else
>>>        /* GCC */
>>>        &&  dict->size * 8<= *pfield_size
>>>  #endif
>>>        &&  (*pbitofs + bitsize)<= *pfield_size) {
>>>        /* continue bit field */
>>>        fieldtype = CONT_BITFIELD;
>>>  #ifndef MS_WIN32
>>>    } else if (bitsize /* this is a bitfield request */
>>>        &&  *pfield_size /* we have a bitfield open */
>>>        &&  dict->size * 8>= *pfield_size
>>>        &&  (*pbitofs + bitsize)<= dict->size * 8) {
>>>        /* expand bit field */
>>>        fieldtype = EXPAND_BITFIELD;
>>>  #endif
>>>
>>
>> So when creating a bitfield structure, it's size can be different on
>> Linux vs Windows.
>>
>> class MyStructure(ctypes.**BigEndianStructure):
>>     _pack_      = 1    # aligned to 8 bits, not ctypes default of 32
>>     _fields_    = [
>>                    ("Data0",   ctypes.c_uint32, 32),
>>                    ("Data1",   ctypes.c_uint8, 3),
>>                    ("Data2",   ctypes.c_uint16, 12),
>>                   ]
>>
>> sizeof for above structure is 6 on GCC build and 7 on MSVC build. This
>> leads to some confusion and issues, because we can't always interop
>> correctly with code compiled with different compiler than the one Python
>> is compiled with on the platform.
>>
>
> Just curious, are you saying that this is the 'cause' of the two bug
> reports, or 'just' something you discovered while investigating them?
>
>
> > Short term solution is to add a warning in the documentation about this.
>
> For 2.7/3.2, yes.
>
>
> > Longer term though, I think it
>
>> would be better to add a property on the Structure class for
>> configurable allocation strategy, for example Native (default), GCC,
>> MSVC and when allocating the bitfield, use given strategy. Native would
>> behave similar to what happens now, but we would also allow GCC-style
>> allocation on Windows for example and the ability to extend this if we
>> ever run into similar issues with other compilers. This shouldn't be too
>> difficult to implement, will be backwards compatible and it would
>> improve interop. I would like to hear some opinions on this.
>>
>
> If this would allow the MSVC-compilied Python to better access dlls
> compiled with gcc (cygwin) on Windows, definitely -- in 3.3.
> If the new feature is (currently) only useful on Windows, doc should say
> so.
>
> --
> Terry Jan Reedy
>
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> riscutiavlad%40gmail.com<http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110625/069cf9d8/attachment.html>