Re: memory-mapped files in Squid

On Thu, 28 Jan 1999, John Dilley wrote:

> Like you say below, most replacement policies will do about as
> well as LRU (plus or minus 10%). The question is whether FIFO will be
> much more efficient -- will it reduce the disk bottleneck?

For writes, almost nothing could be more efficient than a SCAN. Cyclic FS
or FIFO will emulate SCAN in the absence of reads. Of course, random reads
will break the "perfect" order. However, we can still group some writes
together and reduce seek and possibly rotation delays.

> And this is not likely to change, due to the nature of the web
> proxy workload. We and others have found in studies of long-term traces
> that something like 60% of objects are never re-referenced. So most of
> the objects that pass through the cache are garbage (i.e., have no value
> since they are never re-referenced.

Exactly. However, one does not have to write all the objects to disks! If
there is a way to filter out useless objects, we can reduce disk traffic
significantly. We did some "researchy" study on that (published in
CNDS'99), and there are a couple of similar papers (you've mentioned the
study from Poland).

> Why are we not ready for that yet?

The techniques proposed usually go beyond simple algorithms that everybody
got used to. It is not clear how practical and robust those algorithms
are. Humans are usually scared of anything new and controversial.

> I believe there is room for
> improvement of replacement policies (and can say more about this at the
> San Diego workshop, I hope!), and believe there are other opportunities
> as well. Please help me understand if there are fundamental limitations
> to being able to address these, though... Thanks!

IMO, there are no fundamental limitations. It just takes time (and more
research) to move this new category of algorithms from research papers to
practice.

Alex.
Received on Tue Jul 29 2003 - 13:15:56 MDT