Re: Squid, events, and file descriptors

On 23/06/2008, Henrik Nordstrom <henrik_at_henriknordstrom.net> wrote:
> On mån, 2008-06-23 at 12:18 +0100, Bradley Kite wrote:
>
> > I am concerned that, for which ever reason, squid stops processing
> > requests for a particular website, and then fails to detect when
> > clients give up, incorrectly putting the FD into the "half-closed"
> > state, leading to the situation where the client closes the socket but
> > squid still thinks that the socket is open.
>
>
> half-closed state is a bit tricky.. and nearly always the client has
> given up and aborted the connection.
>
> You can set "half_closed_clients off" to make Squid react more promptly
> on those. But it will make a couple obsolete and since long patched
> user-agents fail... It probably won't address the underlying problem
> cause, but probably mask it a bit..
>
>
> > Dropping the squid server out of service on the load balancer to stop
> > actual traffic, and then running "squid -k debug" produces the
> > following messages for lots of different FDs (I presume its for all
> > FD's that squid thinks are active):
>
>
> It's all those half-closed ones..
>
> The fd's that is interesting is the outgoing ones, where Squid is trying
> to connect to the web servers. Or whatever other fd Squid is waiting on.
>
> - external ACL lookups
> - DNS lookups
> - etc,,
>
>
> > I could set "half_closed_clients off", however, even at the start of
> > the decline in file descriptors (ie when there are still file
> > descriptors available) there are problems browsing certain websites,
> > so I think this will just mask a symptom of the problem rather than
> > fix it.
>
>
> Quite likely, but it will also most likely make the problem easier to
> see as you get rid of a lot of sideeffect garbage.
>
>
> > A simple restart of squid fixes the issue, but only for a while. Our
> > support guys are having to restart squid on various devices about 5-10
> > times a day at the moment in order to try minimise impact to our
> > customers.
>
>
> Anyting in /var/log/messages?
>
> it could be as simple as running out of netfilter conntrack entries,
> making it nearly impossible for Squid to make outgoing connections.
>
> Regards
>
> Henrik

Thanks for your ideas so far.

I have added the extra bit of debug as suggested by Adrian, but have
not disabled half-closed clients just yet - as it will cause a
different path of code to execute so the extra debug added wont get
printed out. Once I have the debug I will disable it and see what the
results are.

Regards

--
Brad.