MQTT retained messages: why do I receive an old value on subscribe, and how do I clear it?

When a message is published with the retain flag, the broker delivers it to current subscribers as usual and also stores it as the retained message for that exact topic. Only one retained message exists per topic; a newer retained publish replaces it. Whenever a client later subscribes with a filter matching the topic, the broker sends the stored message straight away, with the retain flag set so the client can tell it is a stored value and not a fresh event.

It has no expiry in MQTT 3.1.1, and a broker with persistence enabled saves it to disk, which is why it survived the restart.

To delete it, publish a retained message with a zero-length payload to the same topic. With the Mosquitto tools: mosquitto_pub -t home/garage/door -r -n, where -r sets retain and -n sends an empty payload. The broker removes the stored message, and new subscribers get nothing for that topic.

To find what is stored, subscribe to everything for a moment: mosquitto_sub -t '#' -v prints the retained messages first, each with its topic. You can then clear the listed topics one by one or with a small script. Recent versions of mosquitto_sub also have a --remove-retained option that clears each retained message it receives; check the manual page of your version.

Use retain for state, not for events. State is something with a current value that a late joiner needs: a door position, a relay state, a device's online/offline status, a configuration value. A new dashboard should see it immediately.

Events and commands should not be retained. A retained "open the door" command is delivered again to the actuator every time it reconnects, which is how doors open by themselves after a router restart. For periodic sensor readings it is a judgement call: retaining gives dashboards a value at start-up, but without a timestamp in the payload nobody can tell that the latest reading is three days old.

That staleness is the underlying issue in your case: the broker faithfully reports the last thing it was told. Combine retained state with a Last Will. If the device registers a retained will on its status topic, the broker marks it offline when it disappears, and consumers can ignore the door state of an offline device. Including a timestamp in the payload achieves the same from the consumer side.

MQTT 5 also offers a message expiry interval, set per message in seconds; the broker discards the retained message when it runs out. It needs a version 5 broker and publisher.

On the receiving side, check the retain flag in the message callback when the distinction matters. In paho-mqtt it is msg.retain: it is true for a stored message delivered on subscription, and normally false for a message that arrives live, even if the publisher set retain on it. An actuator can use that to ignore stored messages at start-up.

MQTT 5 subscriptions also have a retain handling option that tells the broker not to send retained messages at subscribe time at all.