# MQTT Producer Output Plugin This plugin writes metrics to a [MQTT broker][mqtt] acting as a MQTT producer. The plugin supports the MQTT protocols `3.1.1` and `5`. > [!NOTE] > In v2.0.12+ of the mosquitto MQTT server, there is a [bug][mosquitto_bug] > requiring the `keep_alive` value to be set non-zero in Telegraf. Otherwise, > the server will return with `identifier rejected`. > As a reference `eclipse/paho.golang` sets the `keep_alive` to 30. ⭐ Telegraf v0.2.0 🏷️ messaging 💻 all [mqtt]: http://http://mqtt.org/ [mosquitto_bug]: https://github.com/eclipse/mosquitto/issues/2117 ## Global configuration options In addition to the plugin-specific configuration settings, plugins support additional global and plugin configuration settings. These settings are used to modify metrics, tags, and field or create aliases and configure ordering, etc. See the [CONFIGURATION.md][CONFIGURATION.md] for more details. [CONFIGURATION.md]: ../../../docs/CONFIGURATION.md#plugins ## Secret-store support This plugin supports secrets from secret-stores for the `username` and `password` option. See the [secret-store documentation][SECRETSTORE] for more details on how to use them. [SECRETSTORE]: ../../../docs/CONFIGURATION.md#secret-store-secrets ## Configuration ```toml @sample.conf # Configuration for MQTT server to send metrics to [[outputs.mqtt]] ## MQTT Brokers ## The list of brokers should only include the hostname or IP address and the ## port to the broker. This should follow the format `[{scheme}://]{host}:{port}`. For ## example, `localhost:1883` or `mqtt://localhost:1883`. ## Scheme can be any of the following: tcp://, mqtt://, tls://, mqtts:// ## non-TLS and TLS servers can not be mix-and-matched. servers = ["localhost:1883", ] # or ["mqtts://tls.example.com:1883"] ## Protocol can be `3.1.1` or `5`. Default is `3.1.1` # protocol = "3.1.1" ## MQTT Topic for Producer Messages ## MQTT outputs send metrics to this topic format: ## prefix/{{ .Tag "host" }}/{{ .Name }}/{{ .Tag "tag_key" }} ## (e.g. prefix/web01.example.com/mem/some_tag_value) ## Each path segment accepts either a template placeholder, an environment variable, or a tag key ## of the form `{{.Tag "tag_key_name"}}`. All the functions provided by the Sprig library ## (http://masterminds.github.io/sprig/) are available. Empty path elements as well as special MQTT ## characters (such as `+` or `#`) are invalid to form the topic name and will lead to an error. ## In case a tag is missing in the metric, that path segment omitted for the final topic. topic = 'telegraf/{{ .Tag "host" }}/{{ .Name }}' ## QoS policy for messages ## The mqtt QoS policy for sending messages. ## See https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.dev.doc/q029090_.htm ## 0 = at most once ## 1 = at least once ## 2 = exactly once # qos = 2 ## Keep Alive ## Defines the maximum length of time that the broker and client may not ## communicate. Defaults to 0 which turns the feature off. ## ## For version v2.0.12 and later mosquitto there is a bug ## (see https://github.com/eclipse/mosquitto/issues/2117), which requires ## this to be non-zero. As a reference eclipse/paho.mqtt.golang defaults to 30. # keep_alive = 0 ## username and password to connect MQTT server. # username = "telegraf" # password = "metricsmetricsmetricsmetrics" ## client ID ## The unique client id to connect MQTT server. If this parameter is not set ## then a random ID is generated. # client_id = "" ## Timeout for write operations. default: 5s # timeout = "5s" ## Optional TLS Config # tls_ca = "/etc/telegraf/ca.pem" # tls_cert = "/etc/telegraf/cert.pem" # tls_key = "/etc/telegraf/key.pem" ## Use TLS but skip chain & host verification # insecure_skip_verify = false ## When true, metrics will be sent in one MQTT message per flush. Otherwise, ## metrics are written one metric per MQTT message. ## DEPRECATED: Use layout option instead # batch = false ## When true, metric will have RETAIN flag set, making broker cache entries until someone ## actually reads it # retain = false ## Client trace messages ## When set to true, and debug mode enabled in the agent settings, the MQTT ## client's messages are included in telegraf logs. These messages are very ## noisey, but essential for debugging issues. # client_trace = false ## Layout of the topics published. ## The following choices are available: ## non-batch -- send individual messages, one for each metric ## batch -- send all metric as a single message per MQTT topic ## NOTE: The following options will ignore the 'data_format' option and send single values ## field -- send individual messages for each field, appending its name to the metric topic ## homie-v4 -- send metrics with fields and tags according to the 4.0.0 specs ## see https://homieiot.github.io/specification/ # layout = "non-batch" ## HOMIE specific settings ## The following options provide templates for setting the device name ## and the node-ID for the topics. Both options are MANDATORY and can contain ## {{ .Name }} (metric name), {{ .Tag "key"}} (tag reference to 'key') or ## constant strings. The templates MAY NOT contain slashes! # homie_device_name = "" # homie_node_id = "" ## Each data format has its own unique set of configuration options, read ## more about them here: ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md data_format = "influx" ## NOTE: Due to the way TOML is parsed, tables must be at the END of the ## plugin definition, otherwise additional config options are read as part of ## the table ## Optional MQTT 5 publish properties ## These setting only apply if the "protocol" property is set to 5. This must ## be defined at the end of the plugin settings, otherwise TOML will assume ## anything else is part of this table. For more details on publish properties ## see the spec: ## https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901109 # [outputs.mqtt.v5] # content_type = "" # response_topic = "" # message_expiry = "0s" # topic_alias = 0 # [outputs.mqtt.v5.user_properties] # "key1" = "value 1" # "key2" = "value 2" ``` ### `field` layout This layout will publish one topic per metric __field__, only containing the value as string. This means that the `data_format` option will be ignored. For example writing the metrics ```text modbus,location=main\ building,source=device\ 1,status=ok,type=Machine\ A temperature=21.4,serial\ number="324nlk234r5u9834t",working\ hours=123i,supplied=true 1676522982000000000 modbus,location=main\ building,source=device\ 2,status=offline,type=Machine\ B temperature=25.0,supplied=true 1676522982000000000 ``` with configuration ```toml [[outputs.mqtt]] topic = 'telegraf/{{ .Name }}/{{ .Tag "source" }}' layout = "field" ... ``` will result in the following topics and values ```text telegraf/modbus/device 1/temperature 21.4 telegraf/modbus/device 1/serial number 324nlk234r5u9834t telegraf/modbus/device 1/supplied true telegraf/modbus/device 1/working hours 123 telegraf/modbus/device 2/temperature 25 telegraf/modbus/device 2/supplied false ``` __NOTE__: Only fields will be output, tags and the timestamp are omitted. To also output those, please convert them to fields first. ### `homie-v4` layout This layout will publish metrics according to the [Homie v4.0 specification][HomieSpecV4]. Here, the `topic` template will be used to specify the `device-id` path. The __mandatory__ options `homie_device_name` will specify the content of the `$name` topic of the device, while `homie_node_id` will provide a template for the `node-id` part of the topic. Both options can contain [Go templates][GoTemplates] similar to `topic` with `{{ .Name }}` referencing the metric name and `{{ .Tag "key"}}` referencing the tag with the name `key`. [Sprig](http://masterminds.github.io/sprig/) helper functions are available. For example writing the metrics ```text modbus,source=device\ 1,location=main\ building,type=Machine\ A,status=ok temperature=21.4,serial\ number="324nlk234r5u9834t",working\ hours=123i,supplied=true 1676522982000000000 modbus,source=device\ 2,location=main\ building,type=Machine\ B,status=offline supplied=false 1676522982000000000 modbus,source=device\ 2,location=main\ building,type=Machine\ B,status=online supplied=true,Throughput=12345i,Load\ [%]=81.2,account\ no="T3L3GrAf",Temperature=25.38,Voltage=24.1,Current=100 1676542982000000000 ``` with configuration ```toml [[outputs.mqtt]] topic = 'telegraf/{{ .Name }}' layout = "homie-v4" homie_device_name ='{{ .Name }} plugin' homie_node_id = '{{ .Tag "source" }}' ... ``` will result in the following topics and values ```text telegraf/modbus/$homie 4.0 telegraf/modbus/$name modbus plugin telegraf/modbus/$state ready telegraf/modbus/$nodes device-1 telegraf/modbus/device-1/$name device 1 telegraf/modbus/device-1/$properties location,serial-number,source,status,supplied,temperature,type,working-hours telegraf/modbus/device-1/location main building telegraf/modbus/device-1/location/$name location telegraf/modbus/device-1/location/$datatype string telegraf/modbus/device-1/status ok telegraf/modbus/device-1/status/$name status telegraf/modbus/device-1/status/$datatype string telegraf/modbus/device-1/type Machine A telegraf/modbus/device-1/type/$name type telegraf/modbus/device-1/type/$datatype string telegraf/modbus/device-1/source device 1 telegraf/modbus/device-1/source/$name source telegraf/modbus/device-1/source/$datatype string telegraf/modbus/device-1/temperature 21.4 telegraf/modbus/device-1/temperature/$name temperature telegraf/modbus/device-1/temperature/$datatype float telegraf/modbus/device-1/serial-number 324nlk234r5u9834t telegraf/modbus/device-1/serial-number/$name serial number telegraf/modbus/device-1/serial-number/$datatype string telegraf/modbus/device-1/working-hours 123 telegraf/modbus/device-1/working-hours/$name working hours telegraf/modbus/device-1/working-hours/$datatype integer telegraf/modbus/device-1/supplied true telegraf/modbus/device-1/supplied/$name supplied telegraf/modbus/device-1/supplied/$datatype boolean telegraf/modbus/$nodes device-1,device-2 telegraf/modbus/device-2/$name device 2 telegraf/modbus/device-2/$properties location,source,status,supplied,type telegraf/modbus/device-2/location main building telegraf/modbus/device-2/location/$name location telegraf/modbus/device-2/location/$datatype string telegraf/modbus/device-2/status offline telegraf/modbus/device-2/status/$name status telegraf/modbus/device-2/status/$datatype string telegraf/modbus/device-2/type Machine B telegraf/modbus/device-2/type/$name type telegraf/modbus/device-2/type/$datatype string telegraf/modbus/device-2/source device 2 telegraf/modbus/device-2/source/$name source telegraf/modbus/device-2/source/$datatype string telegraf/modbus/device-2/supplied false telegraf/modbus/device-2/supplied/$name supplied telegraf/modbus/device-2/supplied/$datatype boolean telegraf/modbus/device-2/$properties account-no,current,load,location,source,status,supplied,temperature,throughput,type,voltage telegraf/modbus/device-2/location main building telegraf/modbus/device-2/location/$name location telegraf/modbus/device-2/location/$datatype string telegraf/modbus/device-2/status online telegraf/modbus/device-2/status/$name status telegraf/modbus/device-2/status/$datatype string telegraf/modbus/device-2/type Machine B telegraf/modbus/device-2/type/$name type telegraf/modbus/device-2/type/$datatype string telegraf/modbus/device-2/source device 2 telegraf/modbus/device-2/source/$name source telegraf/modbus/device-2/source/$datatype string telegraf/modbus/device-2/temperature 25.38 telegraf/modbus/device-2/temperature/$name Temperature telegraf/modbus/device-2/temperature/$datatype float telegraf/modbus/device-2/voltage 24.1 telegraf/modbus/device-2/voltage/$name Voltage telegraf/modbus/device-2/voltage/$datatype float telegraf/modbus/device-2/current 100 telegraf/modbus/device-2/current/$name Current telegraf/modbus/device-2/current/$datatype float telegraf/modbus/device-2/throughput 12345 telegraf/modbus/device-2/throughput/$name Throughput telegraf/modbus/device-2/throughput/$datatype integer telegraf/modbus/device-2/load 81.2 telegraf/modbus/device-2/load/$name Load [%] telegraf/modbus/device-2/load/$datatype float telegraf/modbus/device-2/account-no T3L3GrAf telegraf/modbus/device-2/account-no/$name account no telegraf/modbus/device-2/account-no/$datatype string telegraf/modbus/device-2/supplied true telegraf/modbus/device-2/supplied/$name supplied telegraf/modbus/device-2/supplied/$datatype boolean ``` #### Important notes and limitations It is important to notice that the __"devices" and "nodes" are dynamically changing__ in Telegraf as the metrics and their structure is not known a-priori. As a consequence, the content of both `$nodes` and `$properties` topics are changing as new `device-id`s, `node-id`s and `properties` (i.e. tags and fields) appear. Best effort is made to limit the number of changes by keeping a superset of all devices and nodes seen, however especially during startup those topics will change more often. Both `topic` and `homie_node_id` should be chosen in a way to group metrics with identical structure! Furthermore, __lifecycle management of devices is very limited__! Devices will only be in `ready` state due to the dynamic nature of Telegraf. Due to limitations in the MQTT client library, it is not possible to set a "will" dynamically. In consequence, devices are only marked `lost` when exiting Telegraf normally and might not change in abnormal aborts. Note that __all field- and tag-names are automatically converted__ to adhere to the [Homie topic ID specification][HomieSpecV4TopicIDs]. In that process, the names are converted to lower-case and forbidden character sequences (everything not being a lower-case character, digit or hyphen) will be replaces by a hyphen. Finally, leading and trailing hyphens are removed. This is important as there is a __risk of name collisions__ between fields and tags of the same node especially after the conversion to ID. Please __make sure to avoid those collisions__ as otherwise property topics will be sent multiple times for the colliding items. [HomieSpecV4]: https://homieiot.github.io/specification/spec-core-v4_0_0 [GoTemplates]: https://pkg.go.dev/text/template [HomieSpecV4TopicIDs]: https://homieiot.github.io/specification/#topic-ids