Re: squid DNS internals confirmation.

On 06/01/14 08:16, Amos Jeffries wrote:
> On 6/01/2014 5:16 p.m., Eliezer Croitoru wrote:
<SNIP>
>>
>> Will the "/etc/hosts" file be loaded before these?
>
> /etc/resolv.conf contains the settings for DNS resolver component.
>
> /etc/hosts contains seed entries for the ipcache/fqdncache components.
>
> They are quite separate and order of configuring does not matter.

OK and it is good to know.

>
>
>>
>> I do see that a "squid -kreconf" will reload the nameserver and hosts
>> settings.
>>
>> I am not sure I will be able to provide a patch that will show the hosts
>> file read progress yet.
>>
>
> It does not seem to matter. See below...
>
Indeed from couple aspects such as too much logs etc.
It is important as a service start-up sequence that needs to be
understood by a human.

>> for example I have tried to access this url
>> "http://postfix.state-of-mind.de/patrick.koetter/smtpauth/building_RPMS_from_SRPMS.html"
>>
>> and got:
>> ##start
>> Unable to determine IP address from host name
>> "postfix.state-of-mind.de"
>>
>> The DNS server returned:
>>
>> No DNS records
>>
>> ##END
>>
>> it took about 15 secs to get this response.
>>
>
> ... This other request to wikimedia shows the problem:
>
>> # tcpdump -n not port 22
>> 07:59:55.008287 IP 192.168.10.121.40062 > 192.168.10.254.53: 46035+ A?
>> www.wikipedia.org. (35)
>> 07:59:55.008292 IP 192.168.10.121.40062 > 192.168.10.254.53: 50532+
>> AAAA? www.wikipedia.org. (35)
>> 07:59:55.024514 IP 192.168.10.254.53 > 192.168.10.121.40062: 50532 3/0/0
>> CNAME wikipedia-lb.wikimedia.org., CNAME text-lb.esams.wikimedia.org., A
>> 91.198.174.192 (116)
>> ##END
>
> - Squid sent *2* DNS requests one for each type of IP.
>
Indeed.

> - 16ms later the first response happens.
>
Indeed.

> - how much longer for the second response?
There wasn't ... at all time I was capturing using tcpdump.

> Current Squid versions send both A and AAAA lookups then wait for *both*
> to return before using the results.
>
> The error pages you mention are an indication that both responses *did*
> come back (eventually) and neither of them contained usable IP addresses
> for Squid to contact.

>
>
>> Now I am a bit unsure about the situation of the test.
>
> So am I even after reading that.
>
> Of all the traces you provided only the tcpdump trace for port 53 when
> lookup up http://www.wikipedia.org/ matches up with anything else you
> are talking about.
>
> Did your tcpdump disgnostic of port 53 cover the entire period until
> well after Squid returned the error page?
Yes but note that it's a test that was repeated more then once and the
daemon was restarted frequently.

>
> What does the dig tool display for www.wikipedia.com when run from the
> Squid machine?
These will shed some light about the issue:
# dig @192.168.10.254 www.wikipedia.org

; <<>> DiG 9.9.3-rpz2+rl.156.01-P2 <<>> @192.168.10.254 www.wikipedia.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36709
;; flags: qr aa rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.wikipedia.org. IN A

;; ANSWER SECTION:
www.wikipedia.org. 0 IN A 91.198.174.192

;; Query time: 0 msec
;; SERVER: 192.168.10.254#53(192.168.10.254)
;; WHEN: Tue Jan 07 02:56:05 IST 2014
;; MSG SIZE rcvd: 51

# dig @127.0.0.1 www.wikipedia.org

; <<>> DiG 9.9.3-rpz2+rl.156.01-P2 <<>> @127.0.0.1 www.wikipedia.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58674
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 13, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.wikipedia.org. IN A

;; ANSWER SECTION:
www.wikipedia.org. 2628 IN CNAME wikipedia-lb.wikimedia.org.
wikipedia-lb.wikimedia.org. 178 IN CNAME text-lb.esams.wikimedia.org.
text-lb.esams.wikimedia.org. 2312 IN A 91.198.174.192

;; AUTHORITY SECTION:
. 36366 IN NS g.root-servers.net.
. 36366 IN NS b.root-servers.net.
. 36366 IN NS m.root-servers.net.
. 36366 IN NS i.root-servers.net.
. 36366 IN NS e.root-servers.net.
. 36366 IN NS a.root-servers.net.
. 36366 IN NS d.root-servers.net.
. 36366 IN NS j.root-servers.net.
. 36366 IN NS k.root-servers.net.
. 36366 IN NS h.root-servers.net.
. 36366 IN NS l.root-servers.net.
. 36366 IN NS c.root-servers.net.
. 36366 IN NS f.root-servers.net.

;; Query time: 4 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Jan 07 02:57:17 IST 2014
;; MSG SIZE rcvd: 338

> There is only one IP address in the tcpdump output, but tcpdump gives
> no indication if it was the IP of the CNAME, or the IP of the master
> nameserver responding to the query.
>
>
>> 1388988544 06/Jan/2014-08:09:04-IST 240 00:00:00:00:00:00
>> 192.168.10.100 TCP_MISS 503 3987 GET
>> http://bits.wikimedia.org/images/wikimedia-button.png HIER_NONE/- text/html
>> 1388988544 06/Jan/2014-08:09:04-IST 404 00:00:00:00:00:00
>> 192.168.10.100 TCP_MISS 200 7760 GET
>> http://upload.wikimedia.org/wikipedia/meta/1/16/MediaWiki-logo_sister_1x.png
>> HIER_DIRECT/91.198.174.208 image/png
>>
>> and in the above I use "host domain.example.com" before squid will
>> contact the service.
>>
>> So my basic idea is to put a name service on the squid machine to allow
>> a more in-depth or a recursive-able dns software to validate the request
>> for squid.
>
> That will only help if the problem is due to response delays or the
> Squid ipcache being too small for the traffic needs.
>
> If the problem is valid responses with only CNAME records, nothing but
> fixing the source DNS zone will help.
A basic assumption which we assume is not the answer.
>
> If the problem is lost packets, finding out where they are lost will
> lead to the proper solution.
>
> Amos
>

I do assume that the only choice now is between iterative and recursive
dns server behaver.
The local dns server behaves like a non-routing friendly system.
I do not like it but the ISP and all other organizations host a DNS
system just because this specific issue do exists.

Thanks,
Eliezer
Received on Mon Jan 06 2014 - 23:14:09 MST