# MQTT: Why the IoT World Runs on a 1999 Protocol

> MQTT moves a temperature reading in about 30 bytes over a connection that survives flaky cellular links and sleepy microcontrollers. The publish/subscribe model, the three QoS levels, and the retained-message and last-will tricks that make it click.

URL: https://harshith.in/blog/mqtt-protocol-internals/
Author: Harshith Sunku (Staff Engineer) — https://harshith.in/about/
Tags: embedded, networking, systems

When I wired an [ESP32 to trigger Wake-on-LAN over MQTT](https://harshith.in/projects/esp32-mqtt-wol/),
the protocol choice justified itself immediately: a microcontroller
with kilobytes of RAM, on Wi-Fi that drops, needed to receive commands
reliably without holding open bespoke connections or polling a server.
MQTT — designed in 1999 for oil-pipeline telemetry over satellite —
solves exactly the constrained-device, unreliable-link problem the
modern IoT rediscovered, and its design decisions are a clinic in
building for scarcity.

## Decouple Everything Through a Broker

HTTP makes the device a client that must know the server, poll for
updates, and re-establish a request per exchange. MQTT inverts it:
every device holds *one* long-lived TCP connection to a central
**broker** (Mosquitto, EMQX, HiveMQ) and communicates only through it,
by **topic**:

```mermaid
flowchart LR
    S1[sensor: temp] -- publish home/kitchen/temp --> B((Broker))
    S2[ESP32: WOL] -- publish home/wol/trigger --> B
    B -- home/kitchen/temp --> A[dashboard]
    B -- home/wol/# --> G[gateway]
```

Publishers and subscribers never know each other exist — they share
only a topic string (`home/kitchen/temp`), a hierarchical path that
subscribers match with wildcards (`+` one level, `#` the rest). This
decoupling is the whole architecture: a new dashboard subscribes
without any device changing; a device publishes without knowing who
listens; the broker fans out. And crucially for constrained devices,
the connection is *bidirectional and persistent* — the ESP32 receives
its WOL command pushed instantly over the connection it already holds,
no polling, no inbound port, no NAT hole to punch.

The efficiency is in the framing: a publish is a 2-byte fixed header,
a short variable header, topic, payload. A sensor reading really is
~30 bytes on the wire — which on metered cellular or a coin-cell
budget is the difference between viable and not.

## QoS: Pay Only for the Reliability You Need

MQTT's three delivery levels are the protocol's most-quoted feature
and its most-misunderstood — the key insight is that they're a
*per-message cost dial*, and higher isn't better, it's more expensive:

- **QoS 0 — at most once.** Fire and forget, one packet, no ack. A
  dropped message is gone. Correct for high-frequency sensor streams
  where the next reading obsoletes this one — paying for delivery
  guarantees on a value you'll replace in a second is waste.
- **QoS 1 — at least once.** PUBLISH → PUBACK, retransmit until
  acked. Guarantees arrival, allows **duplicates** (the ack can be
  lost after delivery — the same ack-crash window every messaging
  system has). Handlers must be idempotent. The sensible default for
  commands and state.
- **QoS 2 — exactly once.** A four-packet handshake
  (PUBLISH/PUBREC/PUBREL/PUBCOMP) that dedupes. Real, but the most
  expensive and slowest — justified only when duplicates genuinely
  cause harm (a billing event, a "dispense once" actuator), and even
  then application-level idempotency is often the more robust answer.

Same "exactly-once is costly, at-least-once + idempotency is usually
right" lesson the whole messaging world converges on — MQTT just
makes it a header field.

## The Two Tricks That Make It Feel Alive

Two features solve problems specific to intermittent devices, and
they're what separate a working IoT system from a frustrating one:

- **Retained messages.** Publish with the retain flag and the broker
  *keeps the last value per topic*, delivering it immediately to any
  future subscriber. So a dashboard connecting at 3pm instantly sees
  the kitchen's last temperature instead of waiting up to a minute
  for the next reading — the broker holds current state so devices
  don't have to be awake when you look.
- **Last Will and Testament.** A client registers a "will" message at
  connect; if it drops *ungracefully* (missed keepalives, not a clean
  disconnect), the broker publishes the will on its behalf —
  typically `home/gateway/status = offline`. This is how you get
  reliable presence/liveness from devices that vanish without
  notice, which is *all* battery and cellular devices. Combined with
  the keepalive ping (the client promises to speak within an
  interval; silence past 1.5× it means dead), it gives you a real
  liveness signal for free.

The mental upgrade MQTT encodes: for unreliable, constrained,
push-oriented systems, a stateful broker holding connections, last
values, and liveness beats stateless request/response — the same
reason message queues exist in the datacenter, applied to devices
that sleep.

## Takeaways

- One persistent broker connection + topic-based pub/sub decouples
  publishers from subscribers and gives constrained devices instant
  push without polling or inbound ports.
- Tiny framing (~2-byte headers) makes it viable on metered/battery
  links where HTTP overhead isn't.
- QoS is a per-message cost dial: 0 for replaceable streams, 1 +
  idempotent handlers as the default, 2 only when duplicates truly
  harm.
- Retained messages give late subscribers current state instantly;
  Last Will + keepalive give reliable presence from devices that
  drop without warning.
- It's a datacenter message-queue's philosophy shrunk to the
  microcontroller — statefulness at the broker so the edge can be
  dumb and asleep.