Re: memory-mapped files in Squid

Hi all

> The idea is to store all objects <=8K into one large memory mapped file.

Don't forget: your memory subsystem will only let you memory-map files that
are comparitively small. I belive that on an Intel system you are limited
to 32bit unsigned... and you can only memory-map one file at a time...
otherwise you exceed various segment sizes.

On linux intel:

void * mmap(void *start, size_t length, int prot , int
       flags, int fd, off_t offset);

So, length's max is max size_t:

linux/types.h:
typedef __kernel_size_t size_t;

/usr/include/asm/posix_types.h:
typedef unsigned int __kernel_size_t;

size is thus 4294967296 - and I bet that you will run into other problems
first.

I believe that something like the CNFS implementation (in concept, not
physical structure) is going to be better. The advantage here is that you
can have 5 files and 5 filedescriptors... you simply 'seek, read' with
these FDs.

> I ran two trace-driven file system load generators, one similar to Squid's
> current file access, and one with a combination of memory-mapped file
> access for <=8K objects and normal file access for >8K objects. The
> results were very encouraging. So I really like to implement something
> like this in Squid and see whether it indeed saves a lot disk I/O.

Can you give me a URL for something like this? I would like to play with
stuff like this too.

Oskar

---
"Haven't slept at all. I don't see why people insist on sleeping. You feel
so much better if you don't. And how can anyone want to lose a minute -
a single minute of being alive?"				-- Think Twice