Populating huge data structures from disk

Michael Bacarella mbac at gpshopper.com
Tue Nov 6 17:36:50 EST 2007


> > Very sure.  If we hit the disk at all performance drops
> > unacceptably.  The application has low locality of reference so
> > on-demand caching isn't an option.  We get the behavior we want when
> > we pre-cache; the issue is simply that it takes so long to build
> > this cache.
>
> The way I do it is run a separate process that mmaps the file and
> reads one byte from each page every half hour or so.  You are right
> that it makes a huge difference.

Why not just disable swap?




More information about the Python-list mailing list