Re: Squid, events, and file descriptors

On Mon, Jun 23, 2008, Bradley Kite wrote:
> Hi all,
>
> I have a problem that I am currently trying to diagnose. Please
> correct me if you think this should be on the -users list instead, but
> I think its a bug and I need further help trying to diagnose it.

Ruh roh!

> Dropping the squid server out of service on the load balancer to stop
> actual traffic, and then running "squid -k debug" produces the
> following messages for lots of different FDs (I presume its for all
> FD's that squid thinks are active):

Ok:

> 2008/06/23 10:57:55| comm_call_handlers(): got fd=11314 read_event=19
> write_event=18 F->read_handler=0x8070890 F->write_handler=(nil)

Ok. hm.

> 2008/06/23 10:57:55| comm_call_handlers(): Calling read handler on fd=11314
> 2008/06/23 10:57:55| clientReadRequest: FD 11314: reading request...
> 2008/06/23 10:57:55| clientReadRequest: FD 11314 closed?

So thats from:

        debug(33, 5) ("clientReadRequest: FD %d closed?\n", fd);
        F->flags.socket_eof = 1;
        conn->defer.until = squid_curtime + 1;
        conn->defer.n++;
        fd_note_static(fd, "half-closed");

> 2008/06/23 10:57:55| cbdataLock: 0x655b16c0
> 2008/06/23 10:57:55| cbdataValid: 0x655b16c0
> 2008/06/23 10:57:55| cbdataUnlock: 0x655b16c0
> 2008/06/23 10:57:55| commSetSelect: FD 11314 type 1
> 2008/06/23 10:57:55| commSetEvents(fd=11314)

I'd modify that debug to include the conn->defer.n value, so you can see what the
heck is going on. The defer count should grow to a fixed amount before the socket
is properly closed.

Try setting half_closed_clients to off?

Ah!

> I could set "half_closed_clients off", however, even at the start of
> the decline in file descriptors (ie when there are still file
> descriptors available) there are problems browsing certain websites,
> so I think this will just mask a symptom of the problem rather than
> fix it.

Like what?

> A simple restart of squid fixes the issue, but only for a while. Our
> support guys are having to restart squid on various devices about 5-10
> times a day at the moment in order to try minimise impact to our
> customers.
>
> We are running squid-2.6-STABLE18 on Linux (Centos 5), using the
> epoll() event notification interface.
>
> Any help will be much appreciated as I've run out of things to check.
> Please let me know if you need any further information.

its hard to tell. The sockets are returning EOF (0 byte reads) and should
be closed. The question is why thats happening..

Adrian

-- 
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -