influx/telegraf
1
0
Fork 0
Code Issues Activity

Adding upstream version 1.34.4.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-05-24 07:26:29 +02:00
parent e393c3af3f
commit 4978089aab
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
4963 changed files with 677545 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

349
plugins/outputs/mqtt/README.md Normal file
View file

@ -0,0 +1,349 @@
# MQTT Producer Output Plugin
This plugin writes metrics to a [MQTT broker][mqtt] acting as a MQTT producer.
The plugin supports the MQTT protocols `3.1.1` and `5`.
> [!NOTE]
> In v2.0.12+ of the mosquitto MQTT server, there is a [bug][mosquitto_bug]
> requiring the `keep_alive` value to be set non-zero in Telegraf. Otherwise,
> the server will return with `identifier rejected`.
> As a reference `eclipse/paho.golang` sets the `keep_alive` to 30.
⭐ Telegraf v0.2.0
🏷️ messaging
💻 all
[mqtt]: http://http://mqtt.org/
[mosquitto_bug]: https://github.com/eclipse/mosquitto/issues/2117
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
In addition to the plugin-specific configuration settings, plugins support
additional global and plugin configuration settings. These settings are used to
modify metrics, tags, and field or create aliases and configure ordering, etc.
See the [CONFIGURATION.md][CONFIGURATION.md] for more details.
[CONFIGURATION.md]: ../../../docs/CONFIGURATION.md#plugins
## Secret-store support
This plugin supports secrets from secret-stores for the `username` and
`password` option.
See the [secret-store documentation][SECRETSTORE] for more details on how
to use them.
[SECRETSTORE]: ../../../docs/CONFIGURATION.md#secret-store-secrets
## Configuration
```toml @sample.conf
# Configuration for MQTT server to send metrics to
[[outputs.mqtt]]
## MQTT Brokers
## The list of brokers should only include the hostname or IP address and the
## port to the broker. This should follow the format `[{scheme}://]{host}:{port}`. For
## example, `localhost:1883` or `mqtt://localhost:1883`.
## Scheme can be any of the following: tcp://, mqtt://, tls://, mqtts://
## non-TLS and TLS servers can not be mix-and-matched.
servers = ["localhost:1883", ] # or ["mqtts://tls.example.com:1883"]
## Protocol can be `3.1.1` or `5`. Default is `3.1.1`
# protocol = "3.1.1"
## MQTT Topic for Producer Messages
## MQTT outputs send metrics to this topic format:
## prefix/{{ .Tag "host" }}/{{ .Name }}/{{ .Tag "tag_key" }}
## (e.g. prefix/web01.example.com/mem/some_tag_value)
## Each path segment accepts either a template placeholder, an environment variable, or a tag key
## of the form `{{.Tag "tag_key_name"}}`. All the functions provided by the Sprig library
## (http://masterminds.github.io/sprig/) are available. Empty path elements as well as special MQTT
## characters (such as `+` or `#`) are invalid to form the topic name and will lead to an error.
## In case a tag is missing in the metric, that path segment omitted for the final topic.
topic = 'telegraf/{{ .Tag "host" }}/{{ .Name }}'
## QoS policy for messages
## The mqtt QoS policy for sending messages.
## See https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.dev.doc/q029090_.htm
## 0 = at most once
## 1 = at least once
## 2 = exactly once
# qos = 2
## Keep Alive
## Defines the maximum length of time that the broker and client may not
## communicate. Defaults to 0 which turns the feature off.
##
## For version v2.0.12 and later mosquitto there is a bug
## (see https://github.com/eclipse/mosquitto/issues/2117), which requires
## this to be non-zero. As a reference eclipse/paho.mqtt.golang defaults to 30.
# keep_alive = 0
## username and password to connect MQTT server.
# username = "telegraf"
# password = "metricsmetricsmetricsmetrics"
## client ID
## The unique client id to connect MQTT server. If this parameter is not set
## then a random ID is generated.
# client_id = ""
## Timeout for write operations. default: 5s
# timeout = "5s"
## Optional TLS Config
# tls_ca = "/etc/telegraf/ca.pem"
# tls_cert = "/etc/telegraf/cert.pem"
# tls_key = "/etc/telegraf/key.pem"
## Use TLS but skip chain & host verification
# insecure_skip_verify = false
## When true, metrics will be sent in one MQTT message per flush. Otherwise,
## metrics are written one metric per MQTT message.
## DEPRECATED: Use layout option instead
# batch = false
## When true, metric will have RETAIN flag set, making broker cache entries until someone
## actually reads it
# retain = false
## Client trace messages
## When set to true, and debug mode enabled in the agent settings, the MQTT
## client's messages are included in telegraf logs. These messages are very
## noisey, but essential for debugging issues.
# client_trace = false
## Layout of the topics published.
## The following choices are available:
## non-batch -- send individual messages, one for each metric
## batch -- send all metric as a single message per MQTT topic
## NOTE: The following options will ignore the 'data_format' option and send single values
## field -- send individual messages for each field, appending its name to the metric topic
## homie-v4 -- send metrics with fields and tags according to the 4.0.0 specs
## see https://homieiot.github.io/specification/
# layout = "non-batch"
## HOMIE specific settings
## The following options provide templates for setting the device name
## and the node-ID for the topics. Both options are MANDATORY and can contain
## {{ .Name }} (metric name), {{ .Tag "key"}} (tag reference to 'key') or
## constant strings. The templates MAY NOT contain slashes!
# homie_device_name = ""
# homie_node_id = ""
## Each data format has its own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
data_format = "influx"
## NOTE: Due to the way TOML is parsed, tables must be at the END of the
## plugin definition, otherwise additional config options are read as part of
## the table
## Optional MQTT 5 publish properties
## These setting only apply if the "protocol" property is set to 5. This must
## be defined at the end of the plugin settings, otherwise TOML will assume
## anything else is part of this table. For more details on publish properties
## see the spec:
## https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901109
# [outputs.mqtt.v5]
# content_type = ""
# response_topic = ""
# message_expiry = "0s"
# topic_alias = 0
# [outputs.mqtt.v5.user_properties]
# "key1" = "value 1"
# "key2" = "value 2"
```
### `field` layout
This layout will publish one topic per metric __field__, only containing the
value as string. This means that the `data_format` option will be ignored.
For example writing the metrics
```text
modbus,location=main\ building,source=device\ 1,status=ok,type=Machine\ A temperature=21.4,serial\ number="324nlk234r5u9834t",working\ hours=123i,supplied=true 1676522982000000000
modbus,location=main\ building,source=device\ 2,status=offline,type=Machine\ B temperature=25.0,supplied=true 1676522982000000000
```
with configuration
```toml
[[outputs.mqtt]]
topic = 'telegraf/{{ .Name }}/{{ .Tag "source" }}'
layout = "field"
...
```
will result in the following topics and values
```text
telegraf/modbus/device 1/temperature 21.4
telegraf/modbus/device 1/serial number 324nlk234r5u9834t
telegraf/modbus/device 1/supplied true
telegraf/modbus/device 1/working hours 123
telegraf/modbus/device 2/temperature 25
telegraf/modbus/device 2/supplied false
```
__NOTE__: Only fields will be output, tags and the timestamp are omitted. To
also output those, please convert them to fields first.
### `homie-v4` layout
This layout will publish metrics according to the
[Homie v4.0 specification][HomieSpecV4]. Here, the `topic` template will be
used to specify the `device-id` path. The __mandatory__ options
`homie_device_name` will specify the content of the `$name` topic of the device,
while `homie_node_id` will provide a template for the `node-id` part of the
topic. Both options can contain [Go templates][GoTemplates] similar to `topic`
with `{{ .Name }}` referencing the metric name and `{{ .Tag "key"}}` referencing
the tag with the name `key`.
[Sprig](http://masterminds.github.io/sprig/) helper functions are available.
For example writing the metrics
```text
modbus,source=device\ 1,location=main\ building,type=Machine\ A,status=ok temperature=21.4,serial\ number="324nlk234r5u9834t",working\ hours=123i,supplied=true 1676522982000000000
modbus,source=device\ 2,location=main\ building,type=Machine\ B,status=offline supplied=false 1676522982000000000
modbus,source=device\ 2,location=main\ building,type=Machine\ B,status=online supplied=true,Throughput=12345i,Load\ [%]=81.2,account\ no="T3L3GrAf",Temperature=25.38,Voltage=24.1,Current=100 1676542982000000000
```
with configuration
```toml
[[outputs.mqtt]]
topic = 'telegraf/{{ .Name }}'
layout = "homie-v4"
homie_device_name ='{{ .Name }} plugin'
homie_node_id = '{{ .Tag "source" }}'
...
```
will result in the following topics and values
```text
telegraf/modbus/$homie 4.0
telegraf/modbus/$name modbus plugin
telegraf/modbus/$state ready
telegraf/modbus/$nodes device-1
telegraf/modbus/device-1/$name device 1
telegraf/modbus/device-1/$properties location,serial-number,source,status,supplied,temperature,type,working-hours
telegraf/modbus/device-1/location main building
telegraf/modbus/device-1/location/$name location
telegraf/modbus/device-1/location/$datatype string
telegraf/modbus/device-1/status ok
telegraf/modbus/device-1/status/$name status
telegraf/modbus/device-1/status/$datatype string
telegraf/modbus/device-1/type Machine A
telegraf/modbus/device-1/type/$name type
telegraf/modbus/device-1/type/$datatype string
telegraf/modbus/device-1/source device 1
telegraf/modbus/device-1/source/$name source
telegraf/modbus/device-1/source/$datatype string
telegraf/modbus/device-1/temperature 21.4
telegraf/modbus/device-1/temperature/$name temperature
telegraf/modbus/device-1/temperature/$datatype float
telegraf/modbus/device-1/serial-number 324nlk234r5u9834t
telegraf/modbus/device-1/serial-number/$name serial number
telegraf/modbus/device-1/serial-number/$datatype string
telegraf/modbus/device-1/working-hours 123
telegraf/modbus/device-1/working-hours/$name working hours
telegraf/modbus/device-1/working-hours/$datatype integer
telegraf/modbus/device-1/supplied true
telegraf/modbus/device-1/supplied/$name supplied
telegraf/modbus/device-1/supplied/$datatype boolean
telegraf/modbus/$nodes device-1,device-2
telegraf/modbus/device-2/$name device 2
telegraf/modbus/device-2/$properties location,source,status,supplied,type
telegraf/modbus/device-2/location main building
telegraf/modbus/device-2/location/$name location
telegraf/modbus/device-2/location/$datatype string
telegraf/modbus/device-2/status offline
telegraf/modbus/device-2/status/$name status
telegraf/modbus/device-2/status/$datatype string
telegraf/modbus/device-2/type Machine B
telegraf/modbus/device-2/type/$name type
telegraf/modbus/device-2/type/$datatype string
telegraf/modbus/device-2/source device 2
telegraf/modbus/device-2/source/$name source
telegraf/modbus/device-2/source/$datatype string
telegraf/modbus/device-2/supplied false
telegraf/modbus/device-2/supplied/$name supplied
telegraf/modbus/device-2/supplied/$datatype boolean
telegraf/modbus/device-2/$properties account-no,current,load,location,source,status,supplied,temperature,throughput,type,voltage
telegraf/modbus/device-2/location main building
telegraf/modbus/device-2/location/$name location
telegraf/modbus/device-2/location/$datatype string
telegraf/modbus/device-2/status online
telegraf/modbus/device-2/status/$name status
telegraf/modbus/device-2/status/$datatype string
telegraf/modbus/device-2/type Machine B
telegraf/modbus/device-2/type/$name type
telegraf/modbus/device-2/type/$datatype string
telegraf/modbus/device-2/source device 2
telegraf/modbus/device-2/source/$name source
telegraf/modbus/device-2/source/$datatype string
telegraf/modbus/device-2/temperature 25.38
telegraf/modbus/device-2/temperature/$name Temperature
telegraf/modbus/device-2/temperature/$datatype float
telegraf/modbus/device-2/voltage 24.1
telegraf/modbus/device-2/voltage/$name Voltage
telegraf/modbus/device-2/voltage/$datatype float
telegraf/modbus/device-2/current 100
telegraf/modbus/device-2/current/$name Current
telegraf/modbus/device-2/current/$datatype float
telegraf/modbus/device-2/throughput 12345
telegraf/modbus/device-2/throughput/$name Throughput
telegraf/modbus/device-2/throughput/$datatype integer
telegraf/modbus/device-2/load 81.2
telegraf/modbus/device-2/load/$name Load [%]
telegraf/modbus/device-2/load/$datatype float
telegraf/modbus/device-2/account-no T3L3GrAf
telegraf/modbus/device-2/account-no/$name account no
telegraf/modbus/device-2/account-no/$datatype string
telegraf/modbus/device-2/supplied true
telegraf/modbus/device-2/supplied/$name supplied
telegraf/modbus/device-2/supplied/$datatype boolean
```
#### Important notes and limitations
It is important to notice that the __"devices" and "nodes" are dynamically
changing__ in Telegraf as the metrics and their structure is not known a-priori.
As a consequence, the content of both `$nodes` and `$properties` topics are
changing as new `device-id`s, `node-id`s and `properties` (i.e. tags and fields)
appear. Best effort is made to limit the number of changes by keeping a
superset of all devices and nodes seen, however especially during startup those
topics will change more often. Both `topic` and `homie_node_id` should be chosen
in a way to group metrics with identical structure!
Furthermore, __lifecycle management of devices is very limited__! Devices will
only be in `ready` state due to the dynamic nature of Telegraf. Due to
limitations in the MQTT client library, it is not possible to set a "will"
dynamically. In consequence, devices are only marked `lost` when exiting
Telegraf normally and might not change in abnormal aborts.
Note that __all field- and tag-names are automatically converted__ to adhere to
the [Homie topic ID specification][HomieSpecV4TopicIDs]. In that process, the
names are converted to lower-case and forbidden character sequences (everything
not being a lower-case character, digit or hyphen) will be replaces by a hyphen.
Finally, leading and trailing hyphens are removed.
This is important as there is a __risk of name collisions__ between fields and
tags of the same node especially after the conversion to ID. Please __make sure
to avoid those collisions__ as otherwise property topics will be sent multiple
times for the colliding items.
[HomieSpecV4]: https://homieiot.github.io/specification/spec-core-v4_0_0
[GoTemplates]: https://pkg.go.dev/text/template
[HomieSpecV4TopicIDs]: https://homieiot.github.io/specification/#topic-ids