[Python-Dev] CHUNKSIZE

[Guido van Rossum]

> Yeah, you're probably right. Hopefully someone knowledgeable on the
> subject will enlighten us.

I bet you wouldn't be able to detect any performance improvement if
the xreadlines() buffer were increased in size.  See the comment in
the source:

/* A larger chunk size doesn't seem to make a difference */
#define CHUNKSIZE  8192

> It does beg the question though:  Why isn't
> readline mmap'd? Would that be more efficient?

(a) File objects are not just used for disk files, but also for
    sockets (via socket.makefile()), pipes, etc.

(b) Mmap APIs vary widely across systems -- we've only got mmap
    support on Unix and Windows.

All this can be addressed by rewriting the I/O system, but that's a
huge job that nobody has volunteered to tackle yet.  (It's
particularly painful because it has to work cross-platform, and beyond
stdio there's nothing we can rely on cross-platform.)

--Guido van Rossum (home page: http://www.python.org/~guido/)