Data Delivery Guarantees
Kamu Node may not have perfect connectivity with MQTT broker, so prolonged periods of disconnects should be accounted for. Additionally, MQTT sources can be used even with kamu-cli tool, and since the tool doesn’t use any background processes it will connect to MQTT broker only when you run:kamu can get when it is connected to the broker depends on:
- Retained message mode
- QoS level of the publisher (e.g. device) and the subscriber (
kamu) - Implementation of your MQTT broker - since delivery semantics is left under-specified in the core MQTT spec.
retained: true, qos: 0 (at most once)- will ensure that broker remembers the last message from a device and always transmits it tokamuupon reconnect. This mode is sufficient for a lot of cases e.g. where device periodically reports temperature data andkamuperiodically ingests the readings on its own schedule.retained: false, qos: 1 (at least once)- will likely make broker to buffer all messages from the device and deliver them tokamuupon reconnect. This mode is good for devices like door sensors, where the whole series of events matters. The exact behavior is very broker-specific, so make sure to read you broker’s documentation.
You can use merge strategies to avoid duplication of retained data in your dataset.