[SciPy-User] help! strange netCDF output

[Ryan May]

On Fri, Nov 12, 2010 at 1:06 PM, Neil Berg <nberg at atmos.ucla.edu> wrote:
>
>>
>> Check that the arrays you're writing out have the same size *data
>> type* as the ones you're creating.  It looks like an int64 vs. int32
>> issue
>>
>> Ryan
>
> You are right.  I changed the data type to "double", which gave me the correct output.  I am still confused on how this solved the problem tho.  Here is what the read-in time_y variable looks like:
<SNIP>
> Thank you for solving the problem, and if you could explain to me more on how this problem was solved I would greatly appreciate it!

This is a bug I hit awhile back in pupynere (on which scipy's netcdf
support is based). Basically, the python netcdf library maps the 'l'
type to NC_INT. However, at least on 64-bit machines, this is a
problem:

>>>print numpy.dtype('l').itemsize
8

So if you use a dtype of 'l', it's creating int64's. However, NetCDF
only supports NC_INT, which always has a size of 4. Somewhere in
there, your array is cast to 8-byte integers. When the netcdf library
goes to write them out, it does (effectively) a basic pointer cast, so
that each of those int64's becomes 2 int32's. Since your original data
were in the range of int32's, the extra byte created in moving to an
int64 just contains 0's, which get written out.

You could probably use a typecode of 'i', which gives you regular
int32's. This would be more space efficient than using a double for a
value with only 4 digits.

Ryan

-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma