MQTT: publish and subscribe

HTTP is a question and answer protocol: the client asks, the server replies, and nothing happens in between. That suits a browser, but it is awkward for sensors. If ten devices must tell a dashboard when something changes, either the dashboard polls all ten constantly, or every device must know the dashboard's address. MQTT solves this by putting a go-between in the middle. Devices only talk to the go-between, and never to each other.

Publish and subscribe

MQTT has three roles:

• A publisher sends a message to a named topic, for example your sensor node.
• A broker is a server that receives every message and forwards it to the right clients. Mosquitto is the most common free one.
• A subscriber tells the broker which topics it wants and receives matching messages, for example a dashboard.

Publishers and subscribers never need to know about each other, which is called decoupling. You can add a second dashboard or a logging service without touching the sensor code.

Messages are tiny. A publish of the value 23.4 to the topic gwordal/node01/temp takes: 2 bytes of fixed header, 2 bytes for the topic length, 19 bytes of topic and 4 bytes of payload:

2 + 2 + 19 + 4 = 27 bytes

An HTTP request and response for the same value is easily 300 bytes of headers. Also, the TCP connection stays open, so there is no handshake per reading. For a battery device that sends often, that difference matters.

Topics

A topic is a path of levels separated by slashes. Design them like a filesystem:

gwordal/node01/temp
gwordal/node01/status
gwordal/node01/led/set

Subscribers can use two wildcards: + matches exactly one level, and # matches everything from that level down. So gwordal/+/temp receives the temperature of every node, and gwordal/node01/# receives everything about node01. Two rules of thumb: put the most general part first, and never start a topic with a slash. Keep commands (.../led/set) and measurements (.../temp) on separate topics, so a device never accidentally receives its own readings as commands.

Quality of Service

MQTT lets each message choose how hard the system tries to deliver it:

For a temperature reading every ten seconds, QoS 0 is right: losing one sample costs nothing, because another arrives soon. For a command like "open the valve", QoS 1 is safer, as long as the action is idempotent: setting the valve to open twice is harmless, toggling it twice is not. QoS 2 is rarely needed on small devices.

Note that the effective QoS is the lower of what the publisher uses and what the subscriber asked for. Also, PubSubClient, the usual Arduino library, publishes only at QoS 0, can subscribe at QoS 0 or 1, and does not support QoS 2.

Retained messages and last will

Two features make MQTT dependable on flaky networks.

A retained message is stored by the broker, and delivered immediately to every new subscriber of that topic. Without it, a dashboard that starts a minute after the sensor published must wait for the next reading. Retain slow-changing state, such as online, firmware version or a setting.

A last will is a message you register with the broker at connect time. If the connection dies without a clean DISCONNECT, for example when the battery dies or Wi-Fi drops, the broker publishes the will for you. Combine both: the will sets status to offline, retained, and on connection the device publishes online, retained. Any client that subscribes sees the true state of the device at once.

The broker detects a dead client with the keep-alive timer. The client promises to send something at least every keepalive seconds, and the broker waits 1.5 x keepalive before declaring it gone. With a keep-alive of 15 s the will fires after 1.5 x 15 s = 22.5 s.

The code

This sketch assumes Wi-Fi is already connected, as in lesson 2. It reconnects without blocking, publishes a temperature every 10 s and listens for a command.

#include <WiFi.h>
#include <PubSubClient.h>

const char* BROKER = "broker.local";            // your Mosquitto host name or IP
const uint16_t PORT = 1883;                     // plain MQTT, unencrypted
const char* TOPIC_TEMP   = "gwordal/node01/temp";
const char* TOPIC_STATUS = "gwordal/node01/status";
const char* TOPIC_CMD    = "gwordal/node01/led/set";
const uint8_t LED_PIN = 2;

String mqttUser, mqttPass;                      // load from Preferences, as in lesson 2

WiFiClient net;
PubSubClient mqtt(net);
uint32_t lastTry = 0, lastPublish = 0;

// Called by mqtt.loop() when a subscribed message arrives
void onMessage(char* topic, byte* payload, unsigned int len) {
  String msg;
  for (unsigned int i = 0; i < len; i++) msg += (char)payload[i];   // payload is not null terminated
  if (String(topic) == TOPIC_CMD) {
    digitalWrite(LED_PIN, msg == "on" ? HIGH : LOW);                // "set" is idempotent
  }
}

bool connectMqtt() {
  // client id, user, pass, will topic, will QoS, will retain, will message
  if (!mqtt.connect("node01", mqttUser.c_str(), mqttPass.c_str(),
                    TOPIC_STATUS, 1, true, "offline")) {
    return false;
  }
  mqtt.publish(TOPIC_STATUS, "online", true);   // retained: new subscribers see it at once
  mqtt.subscribe(TOPIC_CMD, 1);                 // QoS 1 for commands
  return true;
}

void setup() {
  pinMode(LED_PIN, OUTPUT);
  // ... connect Wi-Fi and load mqttUser / mqttPass here ...
  mqtt.setServer(BROKER, PORT);
  mqtt.setCallback(onMessage);
  mqtt.setKeepAlive(15);                        // broker declares us dead after 22.5 s
}

void loop() {
  if (!mqtt.connected()) {
    if (millis() - lastTry > 5000) {            // retry every 5 s, never block loop()
      lastTry = millis();
      connectMqtt();
    }
  } else {
    mqtt.loop();                                // service incoming packets and keep-alive
    if (millis() - lastPublish > 10000) {
      lastPublish = millis();
      char buf[16];
      snprintf(buf, sizeof(buf), "%.1f", 23.4f); // replace with your sensor reading
      mqtt.publish(TOPIC_TEMP, buf);            // QoS 0, not retained
    }
  }
}

You can test it with the Mosquitto command line tools: mosquitto_sub -h broker.local -t "gwordal/#" -v shows everything, and mosquitto_pub -h broker.local -t gwordal/node01/led/set -m on switches the LED.