[Pythonmac-SIG] Re: [Python-Dev] Import hook to do end-of-line conversion?

[Guido van Rossum]

> Different issue.  If this ever gets that far, note that the crunch
> to speed up line-at-a-time file input ended up *requiring* use of
> the native fgets() on Windows, as that was the only way on that
> platform to avoid having the OS do layers of expensive
> multithreading locks for each character read.  So there's no
> efficient way in general to get Windows to recognize \r line endings
> short of implementing our own stdio from the ground up.  On other
> platforms, fileobject.c's get_line() reads one character at a time,
> and I expect its test for "is this an EOL char?" could be
> liberalized at reasonable cost.

I expect that the right solution here is indeed to write our own
stdio-like library from the ground up.  That can solve any number of
problems: telling how many characters are buffered (so you don't have
to use unbuffered mode when using select or poll),
platform-independent line end recognition, and super-efficient
readline() to boot.

But it's a lot of work, and won't be compatible with existing
extensions that use FILE* (not too many I believe).

--Guido van Rossum (home page: http://www.python.org/~guido/)