# DNS Resolution: Every Hop Between You and an IP Address

> A 'simple' name lookup traverses your language runtime, nsswitch, a stub resolver, a recursive resolver, and up to three tiers of authoritative servers — with caches at every layer that disagree. Map the path once and DNS bugs become findable.

URL: https://harshith.in/blog/dns-resolution-path/
Author: Harshith Sunku (Staff Engineer) — https://harshith.in/about/
Tags: networking, systems, linux

"It's always DNS" is a joke with a mechanism behind it: name resolution is
the rare system where a single lookup crosses six administrative domains —
your process, your OS, your router, your ISP or chosen resolver, a TLD, and
the target's DNS host — each with its own cache and failure modes. When it
breaks, people flail because they're debugging a *path* they've never
actually traced. Trace it once, and every DNS incident becomes a
binary-search over layers.

## Layer 0: Before Any Network — Your Own Box

`getaddrinfo()` doesn't do DNS; it does *name service switching*.
`/etc/nsswitch.conf`'s `hosts:` line decides the order — typically
`files dns` or, on systemd distros, `mymachines resolve [!UNAVAIL=return]
files ... dns`. Consequences people rediscover weekly:

- `/etc/hosts` wins over everything (which is both the oldest debugging
  trick and the oldest "why does this box resolve differently" answer).
- `dig` and your application can disagree, *by design*: dig speaks straight
  DNS to whatever's in resolv.conf, while the app went through nsswitch —
  through systemd-resolved's cache, mDNS for `.local`, container-runtime
  magic for service names. "dig works but curl doesn't" is a nsswitch-layer
  statement, not a paradox.
- systemd-resolved adds a stub listener at `127.0.0.53` with its own cache
  and per-interface DNS routing (`resolvectl status` shows the real
  upstreams; `resolvectl query` asks the same path apps use). In
  containers, /etc/resolv.conf is injected by the runtime and points at
  more middleware (kube-dns/CoreDNS — with its own famous quirks like
  `ndots:5` making every external lookup try five cluster suffixes first).

## The Wire: Recursion Downward

Assuming the query escapes your machine, it hits a **recursive resolver**
(ISP, corporate, or 1.1.1.1/8.8.8.8), whose job is the actual walk:

```text
  you ─▶ recursive ─▶ root: "com. is at these 13 letter-servers"    (NS)
                  ─▶ .com: "example.com's NS is ns1.provider.net"   (NS+glue)
                  ─▶ ns1.provider.net: "www.example.com A 93.184.216.34"
      ◀─ answer, cached for TTL
```

`dig +trace www.example.com` performs exactly this walk in front of you —
the single best DNS-education command. In practice the resolver rarely
starts at the root: every intermediate NS referral is cached, so the walk
usually begins at the deepest cached delegation. Also on the wire: UDP with
TCP fallback on truncation (large answers, DNSSEC), EDNS0 buffer-size
negotiation, and — increasingly — the whole first hop wrapped in DoT/DoH,
which moves the privacy boundary but changes nothing else in the model.

## Caching: The Feature That Is Also the Incident

Every layer caches by the record's **TTL**, and every classic DNS incident
is a cache-coherence story:

- **Propagation is a myth** — nothing propagates; caches *expire*. A
  migration cut over "an hour ago" still serves old IPs to whoever cached
  the record 3,599 seconds ago with a 3600 TTL. The professional move is
  lowering TTL to 60 *a day before* the change (you must wait out the
  *old* TTL before the new one applies), then raising it after.
- **Negative caching**: NXDOMAIN answers carry their own TTL (from the
  zone's SOA). Ask for a record *before* creating it and you've cached its
  nonexistence — the deploy-then-it-still-404s classic.
- **Resolvers clamp**: some ignore very low TTLs, some (and some apps —
  the JVM's `networkaddress.cache.ttl` defaulting to *forever* with a
  security manager was legendary) cache far longer than told. End-to-end
  behavior is min/max over layers you don't control.

The debugging ladder, then, follows the path: `resolvectl query` (what apps
see) → `dig @127.0.0.53`, `dig @<upstream>` (whose cache is stale?) →
`dig +trace` (is the authoritative data even right?) → check serials with
`dig SOA` when primaries and secondaries disagree. Ninety percent of "it's
always DNS" resolves to *one layer's cache or one file on the local box* —
the mystique is just the path being invisible.

## Takeaways

- Resolution = nsswitch (files/resolved/mDNS) → stub → recursive → root/TLD
  /authoritative; dig bypasses the first two layers, which is why it can
  "work" while apps fail.
- dig +trace shows the authoritative walk; resolvectl shows the local
  truth on systemd machines; containers add their own resolv.conf layer.
- Nothing propagates — caches expire. Lower TTLs a day ahead of changes,
  and mind negative caching before first-time lookups.
- Layers clamp and override TTLs (resolvers, JVMs); end-to-end freshness is
  the worst cache on the path.
- Debug in path order, one layer per step; DNS mystique is just an untraced
  path.