Populating huge data structures from disk
Michael Bacarella
mbac at gpshopper.com
Tue Nov 6 17:36:50 EST 2007
> > Very sure. If we hit the disk at all performance drops
> > unacceptably. The application has low locality of reference so
> > on-demand caching isn't an option. We get the behavior we want when
> > we pre-cache; the issue is simply that it takes so long to build
> > this cache.
>
> The way I do it is run a separate process that mmaps the file and
> reads one byte from each page every half hour or so. You are right
> that it makes a huge difference.
Why not just disable swap?
More information about the Python-list
mailing list