[Python-Dev] mmap

[Greg Stein]

On Tue, 15 Jun 1999, Guido van Rossum wrote:
> > Another topic: what are the chances of adding the mmap module to the core
> > distribution?  It's restricted to a smallish set of platforms (modern
> > Unices and Win32, I think), but it's quite small, and would be a nice
> > thing to have available in the core, IMHO.  
> 
> If it works on Linux, Solaris, Irix and Windows, and is reasonably
> clean, I'll take it.  Please send it.

Actually, my preference is to see a change to open() rather than a whole
new module. For example, let's say that you open a file, specifying
memory-mapping. Then you create a buffer against that file:

  f = open('foo','rm')  # 'm' means mem-map
  b = buffer(f)
  print b[100:200]

Disclaimer: I haven't looked at the mmap modules (AMK's and Mark's) to see
what capabilities are in there. They may not be expressable soly as open()
changes. (adding add'l params for mmap flags might be another way to
handle this)

I'd like to see mmap native in Python. I won't push, though, until I can
run a test to see what kind of savings will occur when you mmap a .pyc
file and open PyBuffer objects against the thing for the code bytes. My
hypothesis is that you can reduce the working set of Python (i.e. amortize
the cost of a .pyc's code over several processes by mmap'ing it); this
depends on the proportion of code in the pyc relative to "other" stuff.

> > (btw, the buffer object needs more documentation)
> 
> That's for Jack & Greg...

Quite true. My bad :-( ... That would go into the API doc, I guess... I'll
put this on a todo list, but it could be a little while. 

Cheers,
-g

--
Greg Stein, http://www.lyra.org/