Re: html prefetching

Robert Collins wrote:

> This would also allow tuning so fast linked browsers (say users
> on a LAN) don't have prefetching, while users on high latency
> links do..

Hmm.. shouldn't it be the other way around?

Users with low link latency to the cache are the users who benefit most
from reducing the latency from the cache and up. For high latency
clients the request overlap from multiple connections should hide most
of this latency.

The idea behind active prefetching is to reduce the latency of the
client, to make sure his link are fully utilized. The slower link the
client has, the less data needs to be prefetched to be able to fill the
clients connection.

To make this discussion fruitful, concentrate on latency pattern
reduction rather than prefetching. The type of prefetching discussed
here is only one very simple way to approach latency reduction.

I guess it should in most cases be enought to, in priority order:
1. Make sure there is an idle server connection immediately available to
be used when the client request for a inlined object is received and the
object is not in cache.
2. Make sure the client makes use of pipelining where possible, to allow
the proxy to utilize the bandwidth differences without speculating to
much in prefetching / HTML parsing.
3. Tune the proxy to also make use of pipelining to reduce the effects
of latency between the proxy and the server. This is a bit tricky to get
right, but it should be possible. See below.
4. Maybe do a limited prefetching, where retreival are prestarted for a
small number of inlined objects on the page prior to seeing the client
requests. The trick is to know in which order the client will request
the inlined objects, and to prestart a small window of objects while the
client is retreiving the page.

For clients not making use of pipelining, only steps 1 and 4 remains..

-- Issues with proxy -> server pipelining --

a) Retrial of uncompleted requests when the pipe terminates prematurely.
Officially it is not the job of a proxy to retry requests, but in
reality this is often required.
b) Fairness if multiple clients are using the pipeline

To get around this, it is most likely best to limit pipelining to one
client per server connection.

To get around the retrial issues, limit this to the first request
received from the client on a given client connection. Once the client
connection is negotiated to be persistent the client can be assumed to
handle retrials properly.

--
Henrik Nordstrom
Squid hacker