CSV to matrix array

[Oscar Benjamin]

On 13 April 2013 16:30, Ana Dionísio <anadionisio257 at gmail.com> wrote:
> It's still not working. I still have one column with all the data inside, like this:
>
> 2999;T3;3;1;1;Off;ON;OFF;ON;ON;ON;ON;Night;;;;;;
>
> How can I split this data in a way that if I want to print "T3" I would just do "print array[0][1]"?

You initially reported that your data was a CSV file, which normally
means that the values in each row are separated by comma characters
e.g. ','. Actually the data here are separated by semicolons e.g. ';'.
This means that whether you use numpy or the csv module you will need
to specify that the data is separated by semicolons. In numpy you
would do this with

import numpy
data = numpy.loadtxt('file.csv', dtype=int, delimiter=';')

You need to set dtype to be whatever data type you want to convert the
values to e.g. int or float. This is because numpy arrays are
homogeneous. In your case the data (presumably a channel/event header
from an EEG file) is not homogeneous as you have integer data '2999'
followed by the channel name 'T3' which is a string. You can load all
values as strings with

data = numpy.loadtxt('file.csv', dtype=str, delimiter=';')

It is possible to have heterogeneous types in a numpy array using
dtype=object but if you use that with the loadtxt function it will
just use strings for all values.

Alternatively you can use the csv module in the standard library to
load all the data as strings

import csv
with open('file.csv', 'rb') as csvfile:
    data = list(csv.reader(csvfile, delimiter=';'))

This will give you a list of lists of strings rather than a numpy
array. Afterwards you can convert the integer values to int if you
want like so:

for row in data:
    row[0] = int(row[0])

This works because lists can store heterogeneous data, unlike numpy arrays.

Either of the above will let you access the data with e.g. data[2][7]
to get the value from the 8th column of the 3rd row. However, I think
that the better thing to do though would be to use a csv.DictReader
and store your data as a list of dicts. This would look like:

# Change this to names that actually describe each column of your data
columns = ['sample_rate', 'channel_name', 'electrode_number',
'lowpass_filter',...]

data = []
with open('file.csv') as csvfile:
    for row in csv.DictReader(csvfile, fieldnames=columns, delimiter=';'):
        # Convert non-string data here e.g.:
        row['sample_rate'] = int(row['sample_rate'])
        data.append(row)

Now you can access the data using e.g. data[0]['channel_name'] which I
think is better than data[0][1] and you can store data of
heterogeneous type e.g. int, str, etc. in the same row.


Oscar