Reading 'scientific' csv using Pandas?

Martin Schöön martin.schoon at gmail.com
Sun Nov 18 13:22:07 EST 2018


Den 2018-11-18 skrev Shakti Kumar <shakti.shrivastava13 at gmail.com>:
> On Sun, 18 Nov 2018 at 18:18, Martin Schöön <martin.schoon at gmail.com> wrote:
>>
>> Now I hit a bump in the road when some of the data is not in plain
>> decimal notation (xxx,xx) but in 'scientific' (xx,xxxe-xx) notation.
>>
>
> Martin, I believe this should be done by pandas itself while reading
> the csv file,
> I took an example in scientific notation and checked this out,
>
> my sample.csv file is,
> col1,col2
> 1.1,0
> 10.24e-05,1
> 9.492e-10,2
>
That was a quick answer!

My pandas is up to date.

In your example you use the US convention of using "." for decimals
and "," to separate data. This works perfect for me too.

However, my data files use European conventions: decimal "," and TAB
to separate data:

col1	col2
1,1	0
10,24e-05	1
9,492e-10	2

I use 

EUData = pd.read_csv('file.csv', skiprows=1, sep='\t',
decimal=',', engine='python')

to read from such files. This works so so. 'Common floats' (3,1415 etc)
works just fine but 'scientific' stuff (1,6023e23) does not work.

/Martin



More information about the Python-list mailing list