[Numpy-discussion] np.genfromtxt StopIteration Error

[Andras Deak]

Hi Stephen,

Is this not what your original question to this list was about? See
https://mail.python.org/pipermail/numpy-discussion/2019-October/080130.html
and replies.
I still believe that you _can't_ give genfromtxt file names in an
iterable. The iterable input is only inteded to contain the contents
of a single file, probably as if read using `f.readlines()`.
Genfromtxt seems to read data from at most one file with each call.
Regards,

András

On Fri, Oct 11, 2019 at 8:41 PM Stephen P. Molnar
<s.molnar at sbcglobal.net> wrote:
>
> Thanks for the reply.
>
> Keep in mind that i am a Chemist, not an IT person. I used to be a
> marginally proficient FORTRAN II user in the ancient past.
>
> I tried running your code. Please see my comments/questing below:
>
> On 10/11/2019 01:12 PM, Bennet Fauber wrote:
> > I think genfromtxt() wants a filename as the first argument, and you
> > have to tell it the entries in the file are strings not numerics.
> >
> > test.py
> > ------------------------------------------
> > import os
> > import glob
> > import numpy as np
> >
> > fileList = []
> > filesList = []
> >
> > for files in glob.glob("*.log"):
> >      fileName, fileExtension = os.path.splitext(files)
> >      fileList.append(fileName)
> >      filesList.append(files)
> >
> > print('fileList = ', fileList)
> > print('filesList = ', filesList
> >
> > fname = '/tmp/foo.txt'
> There is no '/temp/foo.txt' Where did it come from in your example?
> > print('fname = ', fname)
> > data = np.genfromtxt(fname, dtype=str)
> > print(data)
> > ------------------------------------------
> >
> > Contents of /tmp/foo.txt
> > ------------------------------------------
> > 15-7.log
> > 18-7.log
> > 14-7.log
> > C-VX3.log
> > ------------------------------------------
> >
> > Sample run
> I'm using python 3.7.3, should this make a difference?
> >
> > $ python --version
> > Python 2.7.15+
> >
> > $ python t.py
> > ('fileList = ', ['15-7', '18-7', '14-7', 'C-VX3'])
> > ('filesList = ', ['15-7.log', '18-7.log', '14-7.log', 'C-VX3.log'])
> > ('fname = ', '/tmp/foo.txt')
> > ['15-7.log' '18-7.log' '14-7.log' 'C-VX3.log']
> >
> > Is that any help?
> if I use data = np.genfromtxt('14-7.log', dtype=str, usecols=(1),
> skip_header=27, skip_footer=1, encoding=None) with a specific file name.
> in this example 14-7, I get the resutt I desired:
>
> # 14-7
> -9.960902669
> -8.979504781
> -8.942611364
> -8.915523010
> -8.736508831
> -8.663387139
> -8.410739711
> -8.389146347
> -8.296798909
> -8.168454106
> -8.127990818
> -8.127103774
> -7.979090739
> -7.941872682
> -7.900766215
> -7.881485228
> -7.837826485
> -7.815909505
> -7.722540286
> -7.720346742
>
> so, my question is; why the StopIteration error message in my original
> query? Why is the dcrtipt not iterating over the log files?
>
> Sorry to be so dense.
>
>
> >
> > On Fri, Oct 11, 2019 at 12:41 PM Stephen P. Molnar
> > <s.molnar at sbcglobal.net> wrote:
> >> I have been fighting with the genfromtxt function in numpy for a while now and am trying a slightly different approach.
> >>
> >> Here is the code:
> >>
> >>
> >> import os
> >> import glob
> >> import numpy as np
> >>
> >> fileList = []
> >> filesList = []
> >>
> >> for files in glob.glob("*.log"):
> >> ?????? fileName, fileExtension = os.path.splitext(files)
> >> ?????? fileList.append(fileName)
> >> ?????? filesList.append(files)
> >>
> >> print('fileList = ', fileList)
> >> print('filesList = ', filesList)
> >>
> >> fname = filesList
> >> print('fname = ', fname)
> >> data = np.genfromtxt(fname, usecols=(1), skip_header=27, skip_footer=1, encoding=None)
> >> print(data)
> >>
> >> np.savetxt('fileList.dG', data, fmt='%12.9f', header='${d}')
> >> print(data.dG)
> >>
> >> I am using the Spyder IDE which has a variable explorer which shows:
> >>
> >> filesList = ['C-VX3.log', '18-7.log', '14-7.log', '15-7.log']
> >> fileList = ['C-VX3', '18-7', '14-7', '15-7']
> >>
> >> so the lists that genfromtxt needs are being generated.
> >>
> >> Goggling 'numpy genfromtxt stopiteration error' does not seem to address this problem. At least, I didn't find plaything that I thought applied.
> >>
> >> I would greatly appreciate some assistance here.
> >>
> >> Thanks is advance.
> >>
> >> --
> >> Stephen P. Molnar, Ph.D.
> >> www.molecular-modeling.net
> >> 614.312.7528 (c)
> >> Skype:  smolnar1
> >>
> >> _______________________________________________
> >> NumPy-Discussion mailing list
> >> NumPy-Discussion at python.org
> >> https://mail.python.org/mailman/listinfo/numpy-discussion
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at python.org
> > https://mail.python.org/mailman/listinfo/numpy-discussion
> >
>
> --
> Stephen P. Molnar, Ph.D.
> www.molecular-modeling.net
> 614.312.7528 (c)
> Skype:  smolnar1
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion