[Numpy-discussion] GSOC 2013

[Kurt Smith]

On Wed, Mar 6, 2013 at 4:29 AM, Francesc Alted <francesc at continuum.io> wrote:
> On 3/5/13 7:14 PM, Kurt Smith wrote:
>> On Tue, Mar 5, 2013 at 1:45 AM, Eric Firing <efiring at hawaii.edu> wrote:
>>> On 2013/03/04 9:01 PM, Nicolas Rougier wrote:
>>>>>> This made me think of a serious performance limitation of structured dtypes: a
>>>>>> structured dtype is always "packed", which may lead to terrible byte alignment
>>>>>> for common types.  For instance, `dtype([('a', 'u1'), ('b',
>>>>>> 'u8')]).itemsize == 9`,
>>>>>> meaning that the 8-byte integer is not aligned as an equivalent C-struct's
>>>>>> would be, leading to all sorts of horrors at the cache and register level.
>>> Doesn't the "align" kwarg of np.dtype do what you want?
>>>
>>> In [2]: dt = np.dtype(dict(names=['a', 'b'], formats=['u1', 'u8']),
>>> align=True)
>>>
>>> In [3]: dt.itemsize
>>> Out[3]: 16
>> Thanks!  That's what I get for not checking before posting.
>>
>> Consider this my vote to make `aligned=True` the default.
>
> I would not run too much.  The example above takes 9 bytes to host the
> structure, while a `aligned=True` will take 16 bytes.  I'd rather let
> the default as it is, and in case performance is critical, you can
> always copy the unaligned field to a new (homogeneous) array.

Yes, I can absolutely see the case you're making here, and I made my
"vote" with the understanding that `aligned=False` will almost
certainly stay the default.  Adding 'aligned=True' is simple for me to
do, so no harm done.

My case is based on what's the least surprising behavior: C structs /
all C compilers, the builtin `struct` module, and ctypes `Structure`
subclasses all use padding to ensure aligned fields by default.  You
can turn this off to get packed structures, but the default behavior
in these other places is alignment, which is why I was surprised when
I first saw that NumPy structured dtypes are packed by default.


>
> --
> Francesc Alted
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion