File Read Cache - How to purge?

[Marc 'BlackJack' Rintsch]

On Tue, 21 Aug 2007 02:29:14 -0700, Signal wrote:

>> It (the whole file) is cached by the operating system totally independent
>> of your program, so the memory used does of course not show up in the memory
>> stats of your program... <snip>
> 
> In this case the OS is Windows and monitoring the memory usage in Task
> Manager, not through the script. The entire 759MB file is not being
> cached in memory and only 2MB of memory is used when the script runs.

If you read from a file the operating system usually caches all read data
until there is no cache memory left, then some old cached data is
replaced.  So if you read the whole 756 MB file, even in small blocks, and
have enough RAM chances are that the whole file is in the cache.  And of
course the memory consumption of the process is just the memory for
interpreter, program and data.  2 MB sounds reasonable.

>> > 2. Is there anyway to somehow to take advantage of this "caching" by
>> > initializing it without reading through the entire file first?
>>
>> You mean reading the file without actually reading it!?  :-)
>>
> 
> Think you misunderstood.
> 
> What the "tests" are eluding to is:
> 
> a. The whole file itself is NOT being cached in memory.

Everything read is cached as long as there's enough space in the cache.

> b. If there is mechanism to which it is "caching" something (which
> obviously isn't the whole file itself), why not possibly take
> advantage of it?

How?  Your speedup comes from data in caches, but the time putting it
there was spend in the previous run.  So you only gain something on
subsequent reads on the file.

Ciao,
	Marc 'BlackJack' Rintsch