format number is not recognized in a cycle

[Diego Avesani]

Dear Peter,

thanks a lot for your patience.


this is the code:

import pandas as pd
import numpy as np
from datetime import datetime


#input:
start_date = np.array(["2012-01-01 06:00",'2013-01-01 06:00','2014-01-01 06:00'])
end_date   = np.array(["2013-01-01 05:00",'2014-01-01 05:00','2015-01-01 05:00'])
yearfolder = np.array(['2012','2013','2014'])


for ii in range(0, 1):
   df = pd.read_csv('dati.csv',delimiter=',',header=0,parse_dates=True,na_values=-999)
   df['datatime'] = df['datatime'].map(lambda x: datetime.strptime(str(x), "%Y-%m-%d %H:%M"))
   mask = (df['datatime'] > start_date[ii]) & (df['datatime'] <=  end_date[ii])
   df = df.loc[mask]



and here a piece of my dati.csv file


datatime,T,RH,PSFC,DIR,VEL10,PREC,RAD,CC,FOG
2012-01-01 06:00, -0.1,100, 815,313,  2.6,  0.0,   0,  0,0
2012-01-01 07:00, -1.2, 93, 814,314,  4.8,  0.0,   0,  0,0
2012-01-01 08:00,  1.7, 68, 815,308,  7.5,  0.0,  41, 11,0
2012-01-01 09:00,  2.4, 65, 815,308,  7.4,  0.0, 150, 33,0
2012-01-01 10:00,  3.0, 64, 816,305,  8.4,  0.0, 170, 44,0
2012-01-01 11:00,  2.6, 65, 816,303,  6.3,  0.0, 321, 22,0
2012-01-01 12:00,  2.0, 72, 816,278,  1.3,  0.0, 227, 22,0
2012-01-01 13:00, -0.0, 72, 816,124,  0.1,  0.0, 169, 22,0
2012-01-01 14:00, -0.1, 68, 816,331,  1.4,  0.0, 139, 33,0
2012-01-01 15:00, -4.0, 85, 816,170,  0.6,  0.0,  49,  0,0
2012-01-01 16:00, -5.6, 92, 816,168,  0.8,  0.0,   0,  0,0
2012-01-01 17:00, -6.4, 96, 817,109,  0.4,  0.0,   0,  0,0
2012-01-01 18:00, -6.9,100, 817,116,  0.6,  0.0,   0,  0,0
2012-01-01 19:00, -7.5,100, 817,127,  0.8,  0.0,   0,  0,0
2012-01-01 20:00, -7.7,100, 817,136,  0.6,  0.0,   0,  0,0
2012-01-01 21:00, -7.7,100, 818,118,  0.7,  0.0,   0,  0,0
2012-01-01 22:00, -7.8,100, 817,130,  0.5,  0.0,   0,  0,0
2012-01-01 23:00, -7.9,100, 816,160,  0.6,  0.0,   0,  0,0
2012-01-02 00:00, -8.3,100, 816,123,  0.6,  0.0,   0,  0,0
2012-01-02 01:00, -8.6,100, 815,119,  0.8,  0.0,   0, 11,0
2012-01-02 02:00, -9.1,100, 814,118,  1.1,  0.0,   0, 33,0



thanks, again, a lot

On Monday, 4 February 2019 15:51:41 UTC+1, Peter Otten  wrote:
> Diego Avesani wrote:
> 
> > Dear all,
> > 
> > I have this dataframe:
> > 
> > datatime,T,RH,PSFC,DIR,VEL10,PREC,RAD,CC,FOG
> > 2012-01-01 06:00,  0.4,100, 911,321,  2.5,  0.0,   0,  0,0
> > 2012-01-01 07:00,  0.8,100, 911,198,  0.8,  0.0,   0, 22,0
> > 2012-01-01 08:00,  0.6,100, 912, 44,  1.2,  0.0,  30, 22,0
> > 2012-01-01 09:00,  3.1, 76, 912, 22,  0.8,  0.0, 134, 44,0
> > 2012-01-01 10:00,  3.4, 77, 912, 37,  0.5,  0.0, 191, 67,0
> > 2012-01-01 11:00,  3.5,100, 912,349,  0.4,  0.0, 277, 44,0
> > 2012-01-01 12:00,  3.6,100, 912, 17,  0.9,  0.0, 292, 22,0
> > 2012-01-01 13:00,  3.5,100, 912, 28,  0.3,  0.0, 219, 44,0
> > 2012-01-01 14:00,  3.3, 68, 912, 42,  0.5,  0.0, 151, 22,0
> > 
> > ....
> > ....
> > 
> > I would like to plot and analyse different part of it. Consequently I have
> > created the following variables in order to manage different period:
> > 
> > start_date = np.array(['2012-01-01 06:00','2013-01-01 06:00','2014-01-01
> > 06:00'])
> > end_date   = np.array(['2013-01-01 05:00','2014-01-01 05:00','2015-01-01
> > 05:00'])
> > 
> > However, if I try to perform the following cycle:
> > 
> > for ii in range(0, 1):
> >    pd.read_csv('dati.csv',delimiter=',',header=0,parse_dates=True,
> >    na_values=-999)
> >    df['datatime'] = df['datatime'].map(lambda x: datetime.strptime(str(x),
> >    "%Y-%m-%d %H:%M"))
> >    #
> >    mask = (df['datatime'] > start_date[ii]) & (df['datatime'] <=
> >    end_date[ii])
> > 
> > I get the following error:
> > 
> >     return _get_dtype_type(np.dtype(arr_or_dtype))
> >   File "/usr/lib/python2.7/dist-packages/numpy/core/_internal.py", line
> >   173, in _commastring
> >     (len(result)+1, astr))
> > ValueError: format number 1 of "2012-01-01 06:00" is not recognized
> > 
> > 
> > 
> > On the contrary, if I use the simple variables:
> > start_date = 2012-01-01 06:00
> > end_date   = 2013-01-01 05:00
> > 
> > without the cycle, it works.
> > 
> > I really do not understand why.
> 
> Nor do I.
> 
> Also, I cannot run the code above, and when I fill in the missing parts 
> using guesswork I cannot reproduce the error.
> 
> Would you mind posting a small script, inserted via cut and paste into your 
> message, that we can run without any additions?
> 
> You may want to read http://www.sscce.org/ first.