[Numpy-discussion] Fortran binary files and numpy/scipy

[Ryan May]

Nils Wagner wrote:
> On Mon, 02 Feb 2009 14:07:35 -0600
>   Ryan May <rmay31 at gmail.com> wrote:
>> Nils Wagner wrote:
>>>>> Is this a 64-bit problem ?
>>>>>
>>>> I don't know if it's a 64-bit problem per-se, so much as 
>>>> a disagreement between
>>>> fortran and numpy.  Numpy is making the size of the 
>>>> integer fields 8 bytes, while
>>>> in Fortran, they're only 4 bytes.  When constructing 
>>>> your dtype, use np.int32 or
>>>> '<i4' for your type for the integer fields, and see if 
>>>> that fixes it.
>>>>
>>> dt = 
>>> np.dtype([('isize','int32'),('irow','int32'),('icol','int32'),('value','float')])
>>>
>>>
>>>>>> a
>>> array([(16, 0, 11, 1.2549267404367662e-321),
>>>         (1081065472, 16, 0, 7.9050503334599447e-323),
>>>         (12, 253, 0, 3.4485523805914514e-313),
>>>         (0, 16, 0, 5.3474293932967148e-312),
>>>         (0, 1079312384, 16, 3.3951932655444357e-313), 
>>> (0, 
>>> 14, 251, 62.0),
>>>         (16, 0, 16, 3.1829936864479085e-313),
>>>         (250, 0, 1078525952, 7.9050503334599447e-323),
>>>         (16, 0, 16, 1.2302234581447039e-321),
>>>         (1078231040, 16, 0, 7.9050503334599447e-323),
>>>         (17, 248, 0, 3.4484552433329538e-313),
>>>         (0, 16, 0, 5.2413296037731544e-312),
>>>         (0, 1077805056, 16, 3.3951932655444357e-313), 
>>> (0, 
>>> 19, 246, 27.0),
>>>         (16, 0, 16, 4.2439915819305446e-313),
>>>         (245, 0, 1077411840, 7.9050503334599447e-323)],
>>>        dtype=[('isize', '<i4'), ('irow', '<i4'), 
>>> ('icol', 
>>> '<i4'), ('value', '<f8')])
>>>
>> Maybe on 64-bit machines, the number of bytes is 64-bits 
>> instead of 32 (see the
>> fact that the first 12 bytes of the file are 16 0 11. 
>> Try:
>>
>> dt =
>> np.dtype([('isize','int64'),('irow','int32'),('icol','int32'),('value','float')])
>>
>> Ryan
>>
> 
> Strange
> 
>>>> a
> array([(16, 11, 254, 254.0), (16, 16, 0, 
> 5.3686493512014268e-312),
>         (4638566878703255552, 16, 0, 
> 7.9050503334599447e-323),
>         (1082331758605, 0, 1079312384, 
> 7.9050503334599447e-323),
>         (16, 14, 251, 62.0), (16, 16, 0, 
> 5.3049894774872906e-312),
>         (4632233691727265792, 16, 0, 
> 7.9050503334599447e-323),
>         (1069446856720, 0, 1078231040, 
> 7.9050503334599447e-323),
>         (16, 17, 248, 35.0), (16, 16, 0, 
> 5.2413296037731544e-312),
>         (4629137466983448576, 16, 0, 
> 7.9050503334599447e-323),
>         (1056561954835, 0, 1077608448, 
> 7.9050503334599447e-323),
>         (16, 20, 245, 24.0)],
>        dtype=[('isize', '<i8'), ('irow', '<i4'), ('icol', 
> '<i4'), ('value', '<f8')])
>   

Apparently I was slightly off on the details (it's been awhile since I had to
deal with this nonsense).  The number of bytes written is written before *and*
after writing your actual data.  So the following should work:

dtype=[('isize', '<i8'), ('irow', '<i4'), ('icol', '<i4'), ('value', '<f8'),
'isize2', '<i8'])

Weird.

Ryan

-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma