Re: mmap

--MimeMultipartBoundary
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Stephen R. van den Berg wrote:

> above worst case is equally likely to occur. I.e. there is no
> particular reason why mmapping would cause this worst case to
> happen more often than in the non-mmapping case.

Well, it depends. If we mmap() more memory than we would normally
allocate then we increases our active page set, pushing us closer to
paging.

And if the OS does NOT use VM for file I/O (Linux 2.0 file-writes is one
example, HP-UX another) then using mmap only adds more stress to the VM
system.

If the OS uses VM based file I/O then mmap() could save us a few pages.
The amount is easy to estimate as it is maximum the amount of cache_mem
+ 8K per open swap-object(file).

> Possible. Then again, mmapping large files basically is the same
> operation as using a lot of swap from a kernel point of view. I
> would expect OSes to be quite capable of handling that.

Don't bet on it. HP-UX for example requires a kernel rebuild if you add
large amounts of swap.

Keeping large amounts mmap()ed for a long time is NOT an option for
Squid as this blow any VM system into paging. If we are to use mmap, we
have to do it in small chunks when needed to limit the active VM
pageset. By unmap:ing the pages we don't need we tell the OS that we are
done with these pages. If we don't then the OS pages mmap:ed data and
swap with equal priority, quickly pushing Squid out on swap.

If you don't understand what I am trying to say here, then try to run a
program that mmaps() 100MB of data and see what effect this has on VM
paging when the program uses the data. Then compare this with the same
program using plain read/write or chunked mmaps. I think you will notice
the difference quite fast.

I do expect that many UNIX:es have a lot of trouble handling thousands
of mmaps. If mmap is part of the basic VM design it should be ok, but
where it is "added" I would expect a lot of unexpected troubles and
limitations.

/Henrik

--MimeMultipartBoundary--
Received on Tue Jul 29 2003 - 13:15:50 MDT