Populating huge data structures from disk

[Paul Rubin]

"Michael Bacarella" <mbac at gpshopper.com> writes:
> Very sure.  If we hit the disk at all performance drops
> unacceptably.  The application has low locality of reference so
> on-demand caching isn't an option.  We get the behavior we want when
> we pre-cache; the issue is simply that it takes so long to build
> this cache.

The way I do it is run a separate process that mmaps the file and
reads one byte from each page every half hour or so.  You are right
that it makes a huge difference.