[Tutor] Writing elements of an array to a file using CSV module

[Hanlie Pretorius]

Ok, now I see what I did. Legacy from when the get_rain_data function
returned precipitation and error. I forgot to remove the brackets from
the 'return [precip]' statement.

It works perfectly now. Thanks for all the help.

2010/10/14, Evert Rol <evert.rol at gmail.com>:
>> Thanks for the reply. I've tried your suggestion and am still getting
>> an error (posted below code). Since my posting, I've also added the
>> code to convert the numpy array to a list. My complete script is:
>>
>> -----
>> # -*- coding: utf-8 -*-
>>
>> import pupynere
>> import os
>> import csv
>> import glob
>>
>> def get_file_details(filename):
>>    sub1='3B42.'
>>    sub2='.nc'
>>    extract=filename.split(sub1)[-1].split(sub2)[0]
>>    elements=extract.split('.')
>>    ure={'0':'00:00:00',
>>         '3':'03:00:00',
>>         '6':'06:00:00',
>>         '9':'09:00:00',
>>         '12':'12:00:00',
>>         '15':'15:00:00',
>>         '18':'18:00:00',
>>         '21':'21:00:00'}
>>    date='20'+elements[0]
>>    time=ure[elements[1]]
>>    return[date,time]
>>
>> def get_rain_data(filename):
>>    # trmm blocks covering study area
>>    rows=[87,86,85,86]
>>    cols=[833,833,833,834]
>>
>>    # open file
>>    file = pupynere.netcdf_file(filename,'r')
>>    # read data
>>    precipitation=file.variables['precipitation']
>>    # close file    file.close()
>>
>>    # read values for blocks of interest
>>    precip=precipitation[rows,cols]
>>    # convert numpy array to list
>>    precip=precip.tolist()
>>    return[precip]
>
> You just converted precip to a list (precip.tolist()), then, in the next
> line, you're turning that into a list.
> So you get a list within a list, which results in the errors below. Exactly
> as you hinted at yourself below.
>
> (In fact, if you now would do
>>>> data[0]
> [0.0, 0.0, 0.0, 0.0]
> because this is the representation (repr). Not the 0.0 you previously
> showed.)
>
> Once you get rid of the extra list, you can use either float(data[0]) and %f
> (if you require detailed formatting) or just simply %s, as Alan suggested.
>
>   Evert
>
>
>
>>
>>
>> # get list of .nc files in folder
>> os.chdir('F:\\Hanlie\\UCT\\M.Sc\\Data\\TRMM\\2000\\02_Februarie')
>> filepattern='*.nc'
>> nc_files=glob.glob(filepattern)
>>
>> # write to CSV-file with spaces as delimiter
>> csv.register_dialect('spaces', delimiter=' ')
>> outfilename="trmm_c83a_feb2000.txt"
>> out_text=open(outfilename, 'wb')
>> writer = csv.writer(out_text,dialect='spaces')
>>
>> for file in nc_files:
>>    file_details=get_file_details(file)
>>    data=get_rain_data(file)
>>    writer.writerow(('%s' % (file_details[0]),'%s' %
>> (file_details[1]),'%f' % float(data[0])))
>>
>> out_text.close()
>> -----
>>
>> I now get the error:
>> -----
>> Traceback (most recent call last):
>>  File
>> "F:\Hanlie\UCT\M.Sc\Data\TRMM\lees_netcdf_per_trmm_blok_skryf_vir_pcraster.py",
>> line 76, in <module>
>>    writer.writerow(('%s' % (file_details[0]),'%s' %
>> (file_details[1]),'%f' % float(data[0])))
>> TypeError: float() argument must be a string or a number
>> -----
>>
>> I tried the other suggestion on the list (posted by Alan Gould),
>> namely to replace the %f formatting string with %s in the
>> write.writerow statement:
>> -----
>> writer.writerow( ( file_details[0], file_details[1], str(data[0])  ) )
>> -----
>>
>> This produces a file with lines such as:
>> -----
>> 20000201 00:00:00 "[0.0, 0.0, 0.0, 0.0]"
>> -----
>>
>> I seem to have a list within a list, I'm not sure why. I could write
>> the list elements to variables before I write to the file, but it
>> doesn't seem like a very elegant solution.
>>
>> Can anyone see why I have such a nested data structure? Or is the
>> problem something completely different?
>>
>> Thanks
>> Hanlie
>>
>>
>> 2010/10/14, Evert Rol <evert.rol at gmail.com>:
>>>> I have a numpy array ('data') that is the result of reading a netCDF
>>>> file, and it typically looks like this:
>>>>
>>>> array([ 0.,  0.,  0.,  0.], dtype=float32)
>>>>
>>>>
>>>> I want to write this, after a date and time, to a CSV file, so the CSV
>>>> file would have the entry:
>>>>
>>>>   2000-02-01,09:00,0.0,0.0,0.0,0.0
>>>>
>>>>
>>>> The code I'm using to test it, is:
>>>>
>>>> # file_details[0] is the date and file_details[1] is the time
>>>> writer.writerow(('%s' % (file_details[0]),'%s' %
>>>> (file_details[1]),'%f' % (data[0])))
>>>>
>>>>
>>>> In my opinion, this should give an output line:
>>>>
>>>> 2000-02-01,09:00,0.0
>>>>
>>>>
>>>> Instead, I get an error
>>>>
>>>> TypeError: float argument required, not numpy.ndarray
>>>>
>>>>
>>>>
>>>> Can someone please explain to me why data[0] is giving a numpy array
>>>> rather than one element? I don't understand it, since I get:
>>>>
>>>>>>> data[0]
>>>> 0.0
>>>
>>> This means very little, since a string '0.0' will show the same output
>>> when
>>> printed:
>>>>>> print '0.0'
>>> 0.0
>>>
>>> The exception says 'TypeError', so try:
>>>
>>>>>> type(data[0])
>>> <type 'numpy.float32'>
>>>
>>> Which is indeed what you specified at the start using dtype=float32.
>>>
>>> The standard string formatting doesn't know what to do with a
>>> numpy.float32,
>>> so you'll need to convert it to a float:
>>>
>>>>>> '%f' % float(data[0])
>>> '0.000000'
>>>
>>> (use proper float formatting to get at '0.0', which is what you
>>> apparently
>>> want.)
>>>
>>>
>>>  Evert
>>>
>>>
>
>