[Numpy-discussion] setting decimal accuracy in array operations (scikits.timeseries)

[Christopher Barker]

Robert Kern wrote:
> On Wed, Mar 3, 2010 at 14:09, Marco Tuckner
>> In Python the numbers show:
>> 12.88888888
>>
>> whereas in Excel I see:
>> 12.8888888888888

> If you are still seeing actual calculation differences, we will need
> to see a complete, self-contained example that demonstrates the
> difference.

To add a bit more detail -- unless you are explicitly specifying single 
precision floats (dtype=float32), then both numpy and excel are using 
doubles -- so that's not the source of the differences. Even if you are 
using single precision in numpy, It's pretty rare for that to make a 
significant difference. Something else is going on.

I suspect a different algorithm, you can tell timeseries.convert how you 
want it to interpolate -- who knows what excel is doing.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov