Re: Squid performance wish-list

Stewart Forster wrote:
>Stephen van den Berg wrote:
>> File deletion is a steady background process, not exactly high-priority
>> or highly latency dependent.

>WRONG! Squid deletes as many files as it adds. Since you're also arguing
>for v.large files here, and squid does throw in the odd large file, we're
>talking a disk access for every single block that needs to be deleted.

Well, not quite. A single disk access for every single chunk. Since a file
might very well consist of only one chunk (containing many contiguous
blocks), it's not always so expensive. It depens on the fragmentation
of the filesystem. The more fragmented it becomes, the slower
deletions will work.

>Disk accesses are EXPENSIVE.

True. They become a bit (how much, that depends on a lot of factors) less
expensive if squeezed in between the elevator movement of the head.

> Trivialising the importance of file deletion
>is the first step to having a filesystem whose performance sucks.

I'm not trivialising it, I'm merely pointing out that the requirements of
squid differ in some respects from traditional requirements put on
filesystems.

>16M / 512 = 33768 disk accesses. Going further the direct/indirect system
>requires 1 more disk access per 16M of file size. The linked list system
>requires 32768 disk accesses per extra 16M. Are these numbers big enough
>for you to see why linked lists of blocks are bad?

Yes. Only that in reality, the numbers will not be so bad, as long
as fragmentation is not at it's maximum. I.e. if the 16MB is in one
chunk, then it will take only one disk access to delete it.

>Look, what you're proposing is a filesystem which is basically the same as
>what I've proposed, but you've split out the block pointers across the whole
>file. Kevin and I have explained why this is bad. It is no more complicated

Yes, you have, but apparently you misunderstood the properties of the
chunk in my proposal.

>to code either up. Listen to reason.

Let's just say that I see the merits of your approach, and I'm 97% convinced
that your approach is better than what I proposed.

-- 
Sincerely,                                                          srb@cuci.nl
           Stephen R. van den Berg (AKA BuGless).
This signature third word omitted, yet is comprehensible.