squid DNS internals confirmation.

I am testing couple cases with squid which was slow on the WAN but fast
on the LAN.
The basic issue was that a simple "ping X" has *found* the host dns
record while squid did not for a long time(more then 30 secs).

To make sure that I am going from the ground up and not from the higher
levels towards the lower ones I am verifying the basics.

What would squid Domain Name search would be like?
I see that in the cache.log it first bind a "DNS" sockets one per IP
version.
Then it's adding the nameservers from the local "/etc/resolv.conf" file.

Will the "/etc/hosts" file be loaded before these?

I do see that a "squid -kreconf" will reload the nameserver and hosts
settings.

I am not sure I will be able to provide a patch that will show the hosts
file read progress yet.

for example I have tried to access this url
"http://postfix.state-of-mind.de/patrick.koetter/smtpauth/building_RPMS_from_SRPMS.html"
and got:
##start
ERROR
The requested URL could not be retrieved

The following error was encountered while trying to retrieve the URL:
http://postfix.state-of-mind.de/patrick.koetter/smtpauth/building_RPMS_from_SRPMS.html

     Unable to determine IP address from host name
"postfix.state-of-mind.de"

The DNS server returned:

     No DNS records

This means that the cache was not able to resolve the hostname presented
in the URL. Check if the address is correct.

Your cache administrator is webmaster.
##END

it took about 15 secs to get this response.

I would assume that it's a network issue since the access the local
defined hosts is OK.

Later on I have dumped some packets and found out this(2 dumps):
##START
# tcpdump -n not port 22
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
07:57:04.708222 IP 192.168.10.100.41398 > 192.168.10.121.3128: Flags
[F.], seq 3942284710, ack 347107794, win 307, options [nop,nop,TS val
99531671 ecr 41851070], length 0
07:57:04.708603 IP 192.168.10.121.3128 > 192.168.10.100.41398: Flags
[F.], seq 1, ack 1, win 139, options [nop,nop,TS val 41875608 ecr
99531671], length 0
07:57:04.708969 IP 192.168.10.100.41398 > 192.168.10.121.3128: Flags
[.], ack 2, win 307, options [nop,nop,TS val 99531671 ecr 41875608],
length 0
07:57:09.800015 00:02:6f:9c:bc:0d 802.1B I > ff:ff:ff:ff:ff:ff Null
Unnumbered, test, Flags [Command], length 46
07:57:11.124503 IP 192.168.10.100.41399 > 192.168.10.121.3128: Flags
[P.], seq 540915195:540915564, ack 3267427731, win 248, options
[nop,nop,TS val 99533275 ecr 41846091], length 369
07:57:11.124540 IP 192.168.10.121.3128 > 192.168.10.100.41399: Flags
[.], ack 369, win 130, options [nop,nop,TS val 41882024 ecr 99533275],
length 0
07:57:15.175592 IP 192.168.10.121.3128 > 192.168.10.100.41399: Flags
[.], seq 1:2897, ack 369, win 130, options [nop,nop,TS val 41886075 ecr
99533275], length 2896
07:57:15.175626 IP 192.168.10.121.3128 > 192.168.10.100.41399: Flags
[P.], seq 2897:3941, ack 369, win 130, options [nop,nop,TS val 41886075
ecr 99533275], length 1044
07:57:15.177398 IP 192.168.10.100.41399 > 192.168.10.121.3128: Flags
[.], ack 1449, win 271, options [nop,nop,TS val 99534288 ecr 41886075],
length 0
07:57:15.178651 IP 192.168.10.100.41399 > 192.168.10.121.3128: Flags
[.], ack 2897, win 294, options [nop,nop,TS val 99534288 ecr 41886075],
length 0
07:57:15.179394 IP 192.168.10.100.41399 > 192.168.10.121.3128: Flags
[.], ack 3941, win 316, options [nop,nop,TS val 99534289 ecr 41886075],
length 0
07:57:15.410757 IP 192.168.10.100.41399 > 192.168.10.121.3128: Flags
[P.], seq 369:669, ack 3941, win 316, options [nop,nop,TS val 99534346
ecr 41886075], length 300
07:57:15.410783 IP 192.168.10.121.3128 > 192.168.10.100.41399: Flags
[.], ack 669, win 139, options [nop,nop,TS val 41886310 ecr 99534346],
length 0
07:57:15.411371 IP 192.168.10.121.3128 > 192.168.10.100.41399: Flags
[.], seq 3941:6837, ack 669, win 139, options [nop,nop,TS val 41886311
ecr 99534346], length 2896
07:57:15.411405 IP 192.168.10.121.3128 > 192.168.10.100.41399: Flags
[P.], seq 6837:7810, ack 669, win 139, options [nop,nop,TS val 41886311
ecr 99534346], length 973
07:57:15.413260 IP 192.168.10.100.41399 > 192.168.10.121.3128: Flags
[.], ack 5389, win 339, options [nop,nop,TS val 99534347 ecr 41886311],
length 0
07:57:15.414257 IP 192.168.10.100.41399 > 192.168.10.121.3128: Flags
[.], ack 6837, win 353, options [nop,nop,TS val 99534347 ecr 41886311],
length 0
07:57:15.415256 IP 192.168.10.100.41399 > 192.168.10.121.3128: Flags
[.], ack 7810, win 353, options [nop,nop,TS val 99534348 ecr 41886311],
length 0
07:57:15.420752 IP 192.168.10.100.41399 > 192.168.10.121.3128: Flags
[P.], seq 669:999, ack 7810, win 353, options [nop,nop,TS val 99534349
ecr 41886311], length 330
07:57:15.421315 IP 192.168.10.121.3128 > 192.168.10.100.41399: Flags
[.], seq 7810:10706, ack 999, win 147, options [nop,nop,TS val 41886321
ecr 99534349], length 2896
07:57:15.421350 IP 192.168.10.121.3128 > 192.168.10.100.41399: Flags
[P.], seq 10706:11711, ack 999, win 147, options [nop,nop,TS val
41886321 ecr 99534349], length 1005
07:57:15.424257 IP 192.168.10.100.41399 > 192.168.10.121.3128: Flags
[.], ack 10706, win 353, options [nop,nop,TS val 99534350 ecr 41886321],
length 0
07:57:15.462981 IP 192.168.10.100.41399 > 192.168.10.121.3128: Flags
[.], ack 11711, win 353, options [nop,nop,TS val 99534360 ecr 41886321],
length 0

# tcpdump -n not port 22
07:59:55.008287 IP 192.168.10.121.40062 > 192.168.10.254.53: 46035+ A?
www.wikipedia.org. (35)
07:59:55.008292 IP 192.168.10.121.40062 > 192.168.10.254.53: 50532+
AAAA? www.wikipedia.org. (35)
07:59:55.024514 IP 192.168.10.254.53 > 192.168.10.121.40062: 50532 3/0/0
CNAME wikipedia-lb.wikimedia.org., CNAME text-lb.esams.wikimedia.org., A
91.198.174.192 (116)
##END

While there was a wire query of the DNS record still squid responded
with a simple:
"ERROR
The requested URL could not be retrieved

The following error was encountered while trying to retrieve the URL:
http://www.wikipedia.org/

     Unable to determine IP address from host name "www.wikipedia.org"

The DNS server returned:

     No DNS records

This means that the cache was not able to resolve the hostname presented
in the URL. Check if the address is correct.

Your cache administrator is webmaster.

Generated Mon, 06 Jan 2014 06:00:30 GMT by linux1 (squid/3.4.2)
"

Now I am a bit unsure about the situation of the test.
1388988544 06/Jan/2014-08:09:04-IST 240 00:00:00:00:00:00
192.168.10.100 TCP_MISS 503 3987 GET
http://bits.wikimedia.org/images/wikimedia-button.png HIER_NONE/- text/html
1388988544 06/Jan/2014-08:09:04-IST 404 00:00:00:00:00:00
192.168.10.100 TCP_MISS 200 7760 GET
http://upload.wikimedia.org/wikipedia/meta/1/16/MediaWiki-logo_sister_1x.png
HIER_DIRECT/91.198.174.208 image/png

and in the above I use "host domain.example.com" before squid will
contact the service.

So my basic idea is to put a name service on the squid machine to allow
a more in-depth or a recursive-able dns software to validate the request
for squid.

Thanks,
Eliezer
Received on Mon Jan 06 2014 - 04:16:58 MST