mmap interface in Python

Hrvoje Niksic hniksic at srce.hr
Fri Sep 10 10:31:42 EDT 1999


When discussing Python speed, many people advised me to "slurp the
whole file in".  The only reason why I didn't like that is the obvious 
one -- it burns a lot of memory for large files.

But then I remembered: in C under Unix, I would use mmap() to map the
file to memory, and operate on that memory.  I don't see why the same
thing couldn't be available in Python -- under Unix at least.

To do this, I'd need a `mmap' module that provides a `mmap' function.
The usage would be something like:

import mmap
object = mmap.mmap(fd, length, offset, mmap.PROT_READ, mmap.MAP_SHARED)

For this to be useful, the returned object would have to be able to
pass as string.  For instance, functions like string.find or
string.split, or the re functions should be able to grok it.  A good
addition would be to make mmap objects mutable as an option, but that
is not mandatory.

The objects would get unmapped either with mmap.munmap(), or through
GC.

My question is: is this possible in Python?  If yes, I might give it a 
spin.




More information about the Python-list mailing list