Re: squid rewrite

> From: Henrik Nordstrom <hno@hem.passagen.se>
>
> > I think that squid sooner or later will need to have best of
> > VM and NOVM versions mixed. NOVM has no problem with large
> > objects, and if it uses VM style for objects near average
> > size (10-20K) it can also conserve FD's and keep most hot
> > objects in VM still being able to manage memory usage.
>
> Have anyone made any reliable comparasion on how much performance is
> gained by having a internal hot-objects cache as compared with file
> system? (the objects should be in memory in both cases.. the filesystem
> has it own set of cache pages).

    I'm afraid it is not trivial to make such comparisons, but I can
 speculate theoretically on why some sort of VM could be useful.
 fs cache has its own logic on when and why to through object or parts
 of it out of memory. Same is with VM management - unused ram may be
 freed via paging out. The difference between the fs cache and VM is
 that fs cache operates with fs blocks (4-8KB), while squid via VM can
 operate with distinct bytes. Thus fs cache is either inefficient or
 uses (wastes) very much memory. This might be unnoticed on a dedicated
 cache box, but on a box that services many other things there will be
 a conflict between squid and other parts of system - either squid floods
 fs cache and makes it very inefficient for other tasks or the other way
 round.

    Using some sort of VM based caching allows to avoid disk IO completely
 in ideal case. There might be an configuration option instead of
 max_mem_obj_size that defines how big objects can be kept in VM. As
 average http object size is 2-8KB it is good to be able to have those
 used most handy in memory and avoid disk seeking. Also, it seems bad
 idea to write negative cached objects to disk.
    VM paging involves little overhead apart from disk io for pages in
 question, fs cache miss might include directory searches and inode cache
 misses, thus fs cache miss is much bigger overhead than VM page miss.
    Of course, if having several processes servicing from same disks, it
 is almost impossible to use VM based cache without a major headache...
    
> But I agree that the 1.2 series should only need one.. it should not
> have any of the basic problems that the current trees have (VM or FD
> usage).

    I strongly agree.
    
 Andres Kroonmaa,
 MicroLink Online
 vorguadministraator.
Received on Tue Jul 29 2003 - 13:15:41 MDT