[Numpy-discussion] loadtxt broken if file does not end in newline

[David Huard]

Lisandro,

When you have some time, could you check this patch solves your problem (and
does not introduce new ones) ?

David


Index: numpy/lib/io.py
===================================================================
--- numpy/lib/io.py     (revision 4824)
+++ numpy/lib/io.py     (working copy)
@@ -11,6 +11,7 @@
 import cStringIO
 import tempfile
 import os
+import re

 from cPickle import load as _cload, loads
 from _datasource import DataSource
@@ -291,9 +292,12 @@
             converterseq = [_getconv(dtype.fields[name][0]) \
                             for name in dtype.names]

+    # Remove comments and leading/trailing white space
+    pattern = re.compile(comments)
     for i,line in enumerate(fh):
         if i<skiprows: continue
-        line = line[:line.find(comments)].strip()
+        line = pattern.split(line)[0]
+        line = line.strip()
         if not len(line): continue
         vals = line.split(delimiter)
         if converterseq is None:


2008/2/27, David Huard <david.huard at gmail.com>:
>
> I can look at it.
>
> Would everyone be satisfied with a solution using regular expressions ?
> That is, looking for the following pattern:
>
> pattern = re.compile(r"""
>     ^\s* # leading white space
>     (.*) # Data
>     %s?  # Zero or one comment character
>     (.*) # Comments
>     \s*$ # Trailing white space
>     """%comments, re.VERBOSE)
>
> match = pattern.search(line)
> line, comment = match.groups()
>
> instead of
>
> line = line[:line.find(comments)].strip()
>
> By the way, is there a test function for loadtxt and savetxt ? I couldn't
> find one.
>
>
> David
>
> 2008/2/26, Alan G Isaac <aisaac at american.edu>:
> >
> > On Tue, 26 Feb 2008, Lisandro Dalcin apparently wrote:
> > > I believe the current 'loadtxt' function is broken
> >
> >
> > I agree:
> > <URL:
> > http://projects.scipy.org/pipermail/numpy-discussion/2007-November/030057.html
> > >
> >
> > Cheers,
> >
> > Alan Isaac
> >
> >
> >
> >
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at scipy.org
> > http://projects.scipy.org/mailman/listinfo/numpy-discussion
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20080227/56f692dd/attachment.html>