[Numpy-discussion] ANN: Numpy 1.6.0 beta 2

[Christopher Barker]

On 4/4/11 10:35 PM, Charles R Harris wrote:
>     IIUC, "Ub" is undefined -- "U" means universal newlines, which makes no
>     sense when used with "b" for binary. I looked at the code a ways back,
>     and I can't remember the resolution order, but there isn't any checking
>     for incompatible flags.
>
>     I'd expect that genfromtxt, being txt, and line oriented, should use
>     'rU'. but if it wants the raw line endings (why would it?) then rb
>     should be fine.
>
>
> "U" has been kept around for backwards compatibility, the python
> documentation recommends that it not be used for new code.

That is for  3.*  -- the 2.7.* docs say:

"""
In addition to the standard fopen() values mode may be 'U' or 'rU'. 
Python is usually built with universal newline support; supplying 'U' 
opens the file as a text file, but lines may be terminated by any of the 
following: the Unix end-of-line convention '\n', the Macintosh 
convention '\r', or the Windows convention '\r\n'. All of these external 
representations are seen as '\n' by the Python program. If Python is 
built without universal newline support a mode with 'U' is the same as 
normal text mode. Note that file objects so opened also have an 
attribute called newlines which has a value of None (if no newlines have 
yet been seen), '\n', '\r', '\r\n', or a tuple containing all the 
newline types seen.

Python enforces that the mode, after stripping 'U', begins with 'r', 'w' 
or 'a'.
""

which does, in fact indicate that 'Ub' is NOT allowed. We should be 
using 'Ur', I think. Maybe the "python enforces" is what we saw the 
error from -- it didn't used to enforce anything.


On 4/5/11 7:12 AM, Charles R Harris wrote:

> The 'Ub' mode doesn't work for '\r' on python 3. This may be a bug in
> python, as it works just fine on python 2.7.

"Ub" never made any sense anywhere -- "U" means universal newline text 
file. "b" means binary -- combining them makes no sense. On older 
pythons, the behaviour of 'Ub' was undefined -- now, it looks like it is 
supposed to raise an error.

does 'Ur' work with \r line endings on Python 3?

According to my read of the docs, 'U' does nothing -- "universal" 
newline support is supposed to be the default:

"""
On input, if newline is None, universal newlines mode is enabled. Lines 
in the input can end in '\n', '\r', or '\r\n', and these are translated 
into '\n' before being returned to the caller.
"""

> It may indeed be desirable
> to read the files as text, but that would require more work on both
> loadtxt and genfromtxt.

Why can't we just open the file with mode 'Ur'? text is text, messing 
with line endings shouldn't hurt anything, and it might help.

If we stick with binary, then it comes down to:
- will having an extra \r with Windows files hurt anything? -- probably not.
- Are there many mac-style text files out there anymore? not many.

-Chris




-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov