[SciPy-User] Loading a CSV with loadtxt

[Athanasios Anastasiou]

Hello everyone

Thank you for your answers Esko and Skipper.

It keeps giving me the same error with StringIO (at least its
consistent :-) ). Setting dtype=None equals to passing no dtype at all
in which case, loadtxt assumes that the file is full of floats which
clearly will not work in my case.

The file is a well formed CSV one. It gets loaded into gnumeric
without any problems.

My version of numpy is 1.3.0

I have also tried with a simple one line file containing dummy entries like:
"Άλφα", "Βήτα", 1.0, 0.5, 1200

It still finds that buffer object unreadable :-)

The traceback (or some of it) is:

 501         # Then, view the array using the specified dtype.

--> 502         X = np.array(X, dtype=np.dtype([('', t) for t in dtype_types]))
    503         X = X.view(dtype)
    504     else:

So presumably, the file is read alright, but it's when it comes to
doing that [('', t) for t in dtype_types] that the actual error is
generated.

So i am wondering if i am using the correct names or forms of those
types...Then again...i have also tried with numpy.float and numpy.uint
and it still fails with the same reason :-/

All the best
Athanasios

On Fri, Oct 29, 2010 at 12:28 PM, Esko Lehtonen <esko.lehtonen at iki.fi> wrote:
> 27.10.2010 22:19, Athanasios Anastasiou kirjoitti:
>> Hello everyone
>>
>> I have a CSV file with the following format:
>> Unicode<100, Unicode<100, float, float, uint
>>
>> and i am trying to read it into a numpy array using:
>>
>> theData = numpy.loadtxt('primData.csv',delimiter=',',skiprows=1,dtype={'names':('NodeS','NodeT','flowF','flowB','ID'),'formats':('U100','U100','f','f','uint')})
>>
>> This fails with "expected a readable buffer object". I have also tried
>> first opening the file through codecs as a utf-8 file and passing the
>> file object into loadtxt but i still get the same error. And i have
>> also tried defining the dtype outside of loadtxt and making sure that
>> it works before using it any further.
>>
>> However, when i tried the code that is available at the cookbook:
>> (with some minor modifications)
>> http://www.scipy.org/Cookbook/InputOutput#head-b0366eac0be19c3d7c32fc81c47a7c02508b6f52
>>
>> It works exactly as expected.
>>
>> I don't mind using the latter but i am just wondering what it is that
>> i am doing wrong with loadtxt(?)
>>
>> Any ideas why loadtxt fails in this way?
>
> Hi
>
> I have encountered that error many times. Every time there was something
> wrong with the text file (e.g. a missing or extra comma somewhere,
> especially in the end of line, or maybe an empty line). loadtxt and
> genfromtxt are very picky with the format of files and the error message
> is not very helpful.
>
> - Esko
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>