Return value usage

[MRAB]

Zac Burns wrote:
>> The point of caching is that it lets you retrieve a result cheaply that
>> was expensive to produce by saving the result in case it's needed again.
>> If the caching itself is expensive because it requires network access
>> then, IMHO, that's not proper caching! (You would need a 2-level cache,
>> ie a small local cache to speed up your a larger network-based cache; a
>> cache for a cache.)
>>
> 
> Agreed, to some extent.
> 
> In my application the cache resulted in enormous savings over
> calculating every time - but still several minutes could be saved in
> some common use cases.
> 
> Calculating the values is very expensive indeed (anywhere between
> seconds and minutes each). Getting the cached values takes ~.05sec.
> However, it is common to request 10000 or so calculations - which
> costs ~8min. Of those, perhaps 10% actually use the return value - all
> of them use the side effects. Also the full cache is very large - in
> the hundreds of GB. Having a local cache wouldn't help because
> generally a machine wouldn't request the same cached values twice.
> They are borrowing runtime from other machines.
> 
I'd probably suggest a local cache storing the most recently requested
results, backed up by a larger non-local cache.

You could even record how long it takes to produce each result and how
often or how recently it has been requested; you might decide not to
cache some values in order to reduce the size of the cache if it won't
affect the overall processing time significantly.