[Pythonmac-SIG] Re: [Python-Dev] Import hook to do end-of-line conversion?

[Just van Rossum]

Tim Peters wrote:

> This is a reversal of history.  The code to ignore
>     \r
> when seeing
>     \r\n
> originally (1995) applied to *all* platforms.  I don't know why, but Jack
> submitted a patch to disable this behavior only when "#ifdef macintosh", in
> revision 2.29 of Parser/tokenizer.c, about 4 years ago.  The #ifdef
> introduced then still exists today; 3 lines introduced by that patch start
> with XXX here for clarity (appropriately defined <wink>):

Interesting, I didn't know that. Jack's on holiday now, so he won't be able to
comment for a while.

> I have no idea what Mac C libraries return for text-mode reads.  They must
> convert \r to \n, right? 

Yes.

> In which case I guess any \r remaining *should* be
> "an error" (but where would it come from, if the C library converts all \r
> thingies?).  Do they leave \n alone? 

Nope: \r's get translated to \n and for whatever reason \n's get translated to
\r... So when opening a unix file on the Mac, it will look like it has \r line
endings and when opening a Windows text file on the Mac, it will appear as if it
has \n\r line endings...

> Etc:  submit a patch that makes the
> code above "work", and I'm sure it would be accepted, but a non-Mac person
> can't guess what's needed.

That's probably easy enough -- although would require changing all tokenizer
code that looks for \n to also look for \r, including PyOS_ReadLine(), so it
goes well beyond the snippet you posted. And then there's the Python file
object...

Just