Reading 'scientific' csv using Pandas?

[Shakti Kumar]

On Sun, 18 Nov 2018 at 18:18, Martin Schöön <martin.schoon at gmail.com> wrote:
>
> I am in this project where I try to get an overview of a bunch of
> computer generated (finite element program) data. I have it stored in a
> number of csv files.
>
> Reading the data into spreadsheet programs works fine but is very labour
> intensive so I am working with Pandas in Jupyter notebooks which I find
> much more efficient.
>
> Now I hit a bump in the road when some of the data is not in plain
> decimal notation (xxx,xx) but in 'scientific' (xx,xxxe-xx) notation.
>

Martin, I believe this should be done by pandas itself while reading
the csv file,
I took an example in scientific notation and checked this out,

my sample.csv file is,
col1,col2
1.1,0
10.24e-05,1
9.492e-10,2

and then I execute,
In [29]: a= pd.read_csv('sample.csv')
In [30]: a.values
Out [30]:
array([[1.100e+00, 0.000e+00],
       [1.024e-04, 1.000e+00],
       [9.492e-10, 2.000e+00]])
In [31]: a.values[1][0]
Out[31]: 0.0001024
As you can see, pandas has converted scientific notation to float,
even the data type of these values is numpy.float64

What best I can guess is a problem with your pandas version, there
were some updates with the 0.17.x coming in, maybe give a shot
upgrading your pandas with,
pip install --upgrade pandas
or in case you’re using anaconda then,
conda update pandas

> [snipped for brevity]
> /Martin
> --
> https://mail.python.org/mailman/listinfo/python-list

-- 
Shakti.