The dig Query That Showed Which Resolver Was Lying

I would like to say the stale cache was the problem, because that would be a shorter post, but it started at the some.

The resolver-by-resolver comparison has a single weak property, and here it is: DNS is not one system, it is a chain of caches, and each cache can hold a different answer, so the question is never what the domain resolves to but what it resolves to for this resolver.

The case was a production domain after a record change; the mechanics generalise to any DNS propagation question.

Why the single query was not enough

The query was the dig, and the dig was the local resolver, and the local resolver was the one cache, and the one cache was the not the truth, and the not truth was the misleading, because the local answer was the fine and the user's answer was the stale, and the stale was the different resolver, and the different resolver was the not queried.

The chain is the root and the TLD and the authoritative and the recursive, and the four are the layers, and the layers are the caches, and the caches are the different answers, and the different answers are the inconsistency, and the inconsistency is the user's experience, and the experience is the incident, and the incident is the reason the query must name the resolver.

The authoritative server is the truth, and the truth is the comparison's baseline, and the baseline is the recursive's check, and the check is the stale's detection, and the detection is the fix's entry, and the entry is the two queries, and the two are the authoritative and the recursive, and the difference is the answer.

The same discipline applies to the cutover itself, per the DNS TTL that was still an hour during the migration, and the shared property is the one worth fixing.

The queries that did the work

The authoritative query. The dig against the nameserver is the truth, and the truth is the record's actual value, and the value is the baseline, and the baseline is the fix's reference, and the reference is the discipline, because the recursive query is the cached copy and the copy can be stale.

The recursive query by name. The dig at the resolver is the user's view, and the view is the comparison, and the comparison is the stale's detection, and the detection is the fix, and the fix is the naming, and the naming is the discipline, because the default resolver is the not the user's.

The TTL in the answer. The TTL is the remaining cache life, and the life is the wait, and the wait is the propagation's estimate, and the estimate is the communication, and the communication is the fix, and the fix is the reading, and the reading is the discipline, because the TTL tells you when the stale copy expires.

The trace from the root. The dig with the trace is the delegation's walk, and the walk is the broken delegation's detection, and the detection is the fix, and the fix is the flag, and the flag is the discipline, because the trace shows which layer gives the wrong answer.

The query type check. The record type is the A and the AAAA and the CNAME, and the three are the different answers, and the different answers are the partial failure, and the partial failure is the IPv6's break, and the break is the fix, and the fix is the type's check, and the check is the discipline.

The habit

Ask the authoritative first. The authoritative is the truth, and the truth is the baseline, and the baseline is the comparison, and the comparison is the diagnosis, and the diagnosis is the fix, and the fix is the order, because the recursive first is the cached answer's confusion.

Compare at least three resolvers. The three are the pattern, and the pattern is the widespread or the isolated, and the isolated is the one resolver's cache, and the cache is the wait, and the wait is the fix, and the fix is the comparison, and the comparison is the discipline.

Record the TTL and the time. The TTL and the time are the expiry's calculation, and the calculation is the expectation, and the expectation is the communication, and the communication is the fix, and the fix is the note, and the note is the discipline.

Check the answer, not only the status. The NOERROR with the empty answer is the different from the NXDOMAIN, and the different is the diagnosis, and the diagnosis is the fix, and the fix is the reading, and the reading is the discipline, because the status code alone is the ambiguous.

The rule

DNS resolution depends on which resolver answers, so query the authoritative server for the truth and each recursive resolver for the user's view, read the TTL to know when a stale copy expires, and trace from the root when the delegation itself is suspect.

The site was down for some users and up for others, and one dig against the local resolver showed the correct address while the same query against three public resolvers showed the old one. Get that property wrong and every layer above it lies to you politely.

Published on debugly.dev by Rohit Bhadani | Founder of Krova Cloud