Adding upstream version 1.34.4.
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
parent
e393c3af3f
commit
4978089aab
4963 changed files with 677545 additions and 0 deletions
151
plugins/outputs/iotdb/README.md
Normal file
151
plugins/outputs/iotdb/README.md
Normal file
|
@ -0,0 +1,151 @@
|
|||
# Apache IoTDB Output Plugin
|
||||
|
||||
This plugin writes metrics to an [Apache IoTDB][iotdb] instance, a database
|
||||
for the Internet of Things, supporting session connection and data insertion.
|
||||
|
||||
⭐ Telegraf v1.24.0
|
||||
🏷️ datastore
|
||||
💻 all
|
||||
|
||||
[iotdb]: https://iotdb.apache.org
|
||||
|
||||
## Getting started
|
||||
|
||||
Before using this plugin, please configure the IP address, port number,
|
||||
user name, password and other information of the database server,
|
||||
as well as some data type conversion, time unit and other configurations.
|
||||
|
||||
Please see the [configuration section](#configuration) for an example
|
||||
configuration.
|
||||
|
||||
## Metric Translation
|
||||
|
||||
IoTDB uses a different data format for metric data than telegraf. It is
|
||||
important to note that depending on the metrics being written, the translation
|
||||
may be lossy. This plugin translates to IoTDB format in the following ways:
|
||||
|
||||
### Unsigned Integers
|
||||
|
||||
IoTDB currently **DOES NOT support unsigned integer**.
|
||||
There are three available options of converting uint64, which are specified by
|
||||
setting `uint64_conversion`.
|
||||
|
||||
- `int64_clip`, default option. If an unsigned integer is greater than
|
||||
`math.MaxInt64`, save it as `int64`; else save `math.MaxInt64`
|
||||
(9223372036854775807).
|
||||
- `int64`, force converting an unsigned integer to a`int64`,no mater
|
||||
what the value it is. This option may lead to exception if the value is
|
||||
greater than `int64`.
|
||||
- `text`force converting an unsigned integer to a string, no mater what the
|
||||
value it is.
|
||||
|
||||
### Time Precision
|
||||
|
||||
IoTDB supports a variety of time precision. You can specify which precision
|
||||
you want using the `timestamp_precision` setting. Default is `nanosecond`.
|
||||
Other options are `second`, `millisecond`, `microsecond`.
|
||||
|
||||
### Metadata (tags)
|
||||
|
||||
IoTDB uses a tree model for metadata while Telegraf uses a tag model
|
||||
(see [InfluxDB-Protocol Adapter][InfluxDB-Protocol Adapter]).
|
||||
There are two available options of converting tags, which are specified by
|
||||
setting `convert_tags_to`:
|
||||
|
||||
- `fields`. Treat Tags as measurements. For each Key:Value in Tag,
|
||||
convert them into Measurement, Value, DataType, which are supported in IoTDB.
|
||||
- `device_id`, default option. Treat Tags as part of device id. Tags
|
||||
constitute a subtree of `Name`.
|
||||
|
||||
For example, there is a metric:
|
||||
|
||||
```markdown
|
||||
Name="root.sg.device", Tags={tag1="private", tag2="working"}, Fields={s1=100, s2="hello"}
|
||||
```
|
||||
|
||||
- `fields`, result: `root.sg.device, s1=100, s2="hello", tag1="private", tag2="working"`
|
||||
- `device_id`, result: `root.sg.device.private.working, s1=100, s2="hello"`
|
||||
|
||||
[InfluxDB-Protocol Adapter]: https://iotdb.apache.org/UserGuide/Master/API/InfluxDB-Protocol.html
|
||||
|
||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||
|
||||
In addition to the plugin-specific configuration settings, plugins support
|
||||
additional global and plugin configuration settings. These settings are used to
|
||||
modify metrics, tags, and field or create aliases and configure ordering, etc.
|
||||
See the [CONFIGURATION.md][CONFIGURATION.md] for more details.
|
||||
|
||||
[CONFIGURATION.md]: ../../../docs/CONFIGURATION.md#plugins
|
||||
|
||||
## Secret-store support
|
||||
|
||||
This plugin supports secrets from secret-stores for the `username` and
|
||||
`password` option.
|
||||
See the [secret-store documentation][SECRETSTORE] for more details on how
|
||||
to use them.
|
||||
|
||||
[SECRETSTORE]: ../../../docs/CONFIGURATION.md#secret-store-secrets
|
||||
|
||||
## Configuration
|
||||
|
||||
```toml @sample.conf
|
||||
# Save metrics to an IoTDB Database
|
||||
[[outputs.iotdb]]
|
||||
## Configuration of IoTDB server connection
|
||||
host = "127.0.0.1"
|
||||
# port = "6667"
|
||||
|
||||
## Configuration of authentication
|
||||
# user = "root"
|
||||
# password = "root"
|
||||
|
||||
## Timeout to open a new session.
|
||||
## A value of zero means no timeout.
|
||||
# timeout = "5s"
|
||||
|
||||
## Configuration of type conversion for 64-bit unsigned int
|
||||
## IoTDB currently DOES NOT support unsigned integers (version 13.x).
|
||||
## 32-bit unsigned integers are safely converted into 64-bit signed integers by the plugin,
|
||||
## however, this is not true for 64-bit values in general as overflows may occur.
|
||||
## The following setting allows to specify the handling of 64-bit unsigned integers.
|
||||
## Available values are:
|
||||
## - "int64" -- convert to 64-bit signed integers and accept overflows
|
||||
## - "int64_clip" -- convert to 64-bit signed integers and clip the values on overflow to 9,223,372,036,854,775,807
|
||||
## - "text" -- convert to the string representation of the value
|
||||
# uint64_conversion = "int64_clip"
|
||||
|
||||
## Configuration of TimeStamp
|
||||
## TimeStamp is always saved in 64bits int. timestamp_precision specifies the unit of timestamp.
|
||||
## Available value:
|
||||
## "second", "millisecond", "microsecond", "nanosecond"(default)
|
||||
# timestamp_precision = "nanosecond"
|
||||
|
||||
## Handling of tags
|
||||
## Tags are not fully supported by IoTDB.
|
||||
## A guide with suggestions on how to handle tags can be found here:
|
||||
## https://iotdb.apache.org/UserGuide/Master/API/InfluxDB-Protocol.html
|
||||
##
|
||||
## Available values are:
|
||||
## - "fields" -- convert tags to fields in the measurement
|
||||
## - "device_id" -- attach tags to the device ID
|
||||
##
|
||||
## For Example, a metric named "root.sg.device" with the tags `tag1: "private"` and `tag2: "working"` and
|
||||
## fields `s1: 100` and `s2: "hello"` will result in the following representations in IoTDB
|
||||
## - "fields" -- root.sg.device, s1=100, s2="hello", tag1="private", tag2="working"
|
||||
## - "device_id" -- root.sg.device.private.working, s1=100, s2="hello"
|
||||
# convert_tags_to = "device_id"
|
||||
|
||||
## Handling of unsupported characters
|
||||
## Some characters in different versions of IoTDB are not supported in path name
|
||||
## A guide with suggetions on valid paths can be found here:
|
||||
## for iotdb 0.13.x -> https://iotdb.apache.org/UserGuide/V0.13.x/Reference/Syntax-Conventions.html#identifiers
|
||||
## for iotdb 1.x.x and above -> https://iotdb.apache.org/UserGuide/V1.3.x/User-Manual/Syntax-Rule.html#identifier
|
||||
##
|
||||
## Available values are:
|
||||
## - "1.0", "1.1", "1.2", "1.3" -- use backticks to enclose tags with forbidden characters
|
||||
## such as @$#:[]{}() and space
|
||||
## - "0.13" -- use backticks to enclose tags with forbidden characters
|
||||
## such as space
|
||||
## Keep this section commented if you don't want to sanitize the path
|
||||
# sanitize_tag = "1.3"
|
||||
```
|
346
plugins/outputs/iotdb/iotdb.go
Normal file
346
plugins/outputs/iotdb/iotdb.go
Normal file
|
@ -0,0 +1,346 @@
|
|||
//go:generate ../../../tools/readme_config_includer/generator
|
||||
package iotdb
|
||||
|
||||
import (
|
||||
_ "embed"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/apache/iotdb-client-go/client"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/config"
|
||||
"github.com/influxdata/telegraf/internal/choice"
|
||||
"github.com/influxdata/telegraf/plugins/outputs"
|
||||
)
|
||||
|
||||
//go:embed sample.conf
|
||||
var sampleConfig string
|
||||
|
||||
// matches any word that has a non valid backtick
|
||||
// `word` <- doesn't match
|
||||
// “word , `wo`rd` , `word , word` <- match
|
||||
var forbiddenBacktick = regexp.MustCompile("^[^\x60].*?[\x60]+.*?[^\x60]$|^[\x60].*[\x60]+.*[\x60]$|^[\x60]+.*[^\x60]$|^[^\x60].*[\x60]+$")
|
||||
var allowedBacktick = regexp.MustCompile("^[\x60].*[\x60]$")
|
||||
|
||||
type IoTDB struct {
|
||||
Host string `toml:"host"`
|
||||
Port string `toml:"port"`
|
||||
User config.Secret `toml:"user"`
|
||||
Password config.Secret `toml:"password"`
|
||||
Timeout config.Duration `toml:"timeout"`
|
||||
ConvertUint64To string `toml:"uint64_conversion"`
|
||||
TimeStampUnit string `toml:"timestamp_precision"`
|
||||
TreatTagsAs string `toml:"convert_tags_to"`
|
||||
SanitizeTags string `toml:"sanitize_tag"`
|
||||
Log telegraf.Logger `toml:"-"`
|
||||
|
||||
sanityRegex []*regexp.Regexp
|
||||
session *client.Session
|
||||
}
|
||||
|
||||
type recordsWithTags struct {
|
||||
// IoTDB Records basic data struct
|
||||
DeviceIDList []string
|
||||
MeasurementsList [][]string
|
||||
ValuesList [][]interface{}
|
||||
DataTypesList [][]client.TSDataType
|
||||
TimestampList []int64
|
||||
// extra tags
|
||||
TagsList [][]*telegraf.Tag
|
||||
}
|
||||
|
||||
func (*IoTDB) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
// Init is for setup, and validating config.
|
||||
func (s *IoTDB) Init() error {
|
||||
if s.Timeout < 0 {
|
||||
return errors.New("negative timeout")
|
||||
}
|
||||
if !choice.Contains(s.ConvertUint64To, []string{"int64", "int64_clip", "text"}) {
|
||||
return fmt.Errorf("unknown 'uint64_conversion' method %q", s.ConvertUint64To)
|
||||
}
|
||||
if !choice.Contains(s.TimeStampUnit, []string{"second", "millisecond", "microsecond", "nanosecond"}) {
|
||||
return fmt.Errorf("unknown 'timestamp_precision' method %q", s.TimeStampUnit)
|
||||
}
|
||||
if !choice.Contains(s.TreatTagsAs, []string{"fields", "device_id"}) {
|
||||
return fmt.Errorf("unknown 'convert_tags_to' method %q", s.TreatTagsAs)
|
||||
}
|
||||
|
||||
if s.User.Empty() {
|
||||
s.User.Destroy()
|
||||
s.User = config.NewSecret([]byte("root"))
|
||||
}
|
||||
if s.Password.Empty() {
|
||||
s.Password.Destroy()
|
||||
s.Password = config.NewSecret([]byte("root"))
|
||||
}
|
||||
|
||||
switch s.SanitizeTags {
|
||||
case "0.13":
|
||||
matchUnsupportedCharacter := regexp.MustCompile("[^0-9a-zA-Z_:@#${}\x60]")
|
||||
|
||||
regex := []*regexp.Regexp{matchUnsupportedCharacter}
|
||||
s.sanityRegex = append(s.sanityRegex, regex...)
|
||||
|
||||
// from version 1.x.x IoTDB changed the allowed keys in nodes
|
||||
case "1.0", "1.1", "1.2", "1.3":
|
||||
matchUnsupportedCharacter := regexp.MustCompile("[^0-9a-zA-Z_\x60]")
|
||||
matchNumericString := regexp.MustCompile(`^\d+$`)
|
||||
|
||||
regex := []*regexp.Regexp{matchUnsupportedCharacter, matchNumericString}
|
||||
s.sanityRegex = append(s.sanityRegex, regex...)
|
||||
}
|
||||
|
||||
s.Log.Info("Initialization completed.")
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *IoTDB) Connect() error {
|
||||
username, err := s.User.Get()
|
||||
if err != nil {
|
||||
return fmt.Errorf("getting username failed: %w", err)
|
||||
}
|
||||
password, err := s.Password.Get()
|
||||
if err != nil {
|
||||
username.Destroy()
|
||||
return fmt.Errorf("getting password failed: %w", err)
|
||||
}
|
||||
defer password.Destroy()
|
||||
sessionConf := &client.Config{
|
||||
Host: s.Host,
|
||||
Port: s.Port,
|
||||
UserName: username.String(),
|
||||
Password: password.String(),
|
||||
}
|
||||
username.Destroy()
|
||||
password.Destroy()
|
||||
|
||||
var ss = client.NewSession(sessionConf)
|
||||
s.session = &ss
|
||||
timeoutInMs := int(time.Duration(s.Timeout).Milliseconds())
|
||||
if err := s.session.Open(false, timeoutInMs); err != nil {
|
||||
return fmt.Errorf("connecting to %s:%s failed: %w", s.Host, s.Port, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *IoTDB) Close() error {
|
||||
return s.session.Close()
|
||||
}
|
||||
|
||||
// Write should write immediately to the output, and not buffer writes
|
||||
// (Telegraf manages the buffer for you). Returning an error will fail this
|
||||
// batch of writes and the entire batch will be retried automatically.
|
||||
func (s *IoTDB) Write(metrics []telegraf.Metric) error {
|
||||
// Convert Metrics to Records with Tags
|
||||
rwt, err := s.convertMetricsToRecordsWithTags(metrics)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// Write to client.
|
||||
// If first writing fails, the client will automatically retry three times. If all fail, it returns an error.
|
||||
if err := s.writeRecordsWithTags(rwt); err != nil {
|
||||
return fmt.Errorf("write failed: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Find out data type of the value and return it's id in TSDataType, and convert it if necessary.
|
||||
func (s *IoTDB) getDataTypeAndValue(value interface{}) (client.TSDataType, interface{}) {
|
||||
switch v := value.(type) {
|
||||
case int32:
|
||||
return client.INT32, v
|
||||
case int64:
|
||||
return client.INT64, v
|
||||
case uint32:
|
||||
return client.INT64, int64(v)
|
||||
case uint64:
|
||||
switch s.ConvertUint64To {
|
||||
case "int64_clip":
|
||||
if v <= uint64(math.MaxInt64) {
|
||||
return client.INT64, int64(v)
|
||||
}
|
||||
return client.INT64, int64(math.MaxInt64)
|
||||
case "int64":
|
||||
return client.INT64, int64(v)
|
||||
case "text":
|
||||
return client.TEXT, strconv.FormatUint(v, 10)
|
||||
default:
|
||||
return client.UNKNOWN, int64(0)
|
||||
}
|
||||
case float64:
|
||||
return client.DOUBLE, v
|
||||
case string:
|
||||
return client.TEXT, v
|
||||
case bool:
|
||||
return client.BOOLEAN, v
|
||||
default:
|
||||
return client.UNKNOWN, int64(0)
|
||||
}
|
||||
}
|
||||
|
||||
// convert Timestamp Unit according to config
|
||||
func (s *IoTDB) convertTimestampOfMetric(m telegraf.Metric) (int64, error) {
|
||||
switch s.TimeStampUnit {
|
||||
case "second":
|
||||
return m.Time().Unix(), nil
|
||||
case "millisecond":
|
||||
return m.Time().UnixMilli(), nil
|
||||
case "microsecond":
|
||||
return m.Time().UnixMicro(), nil
|
||||
case "nanosecond":
|
||||
return m.Time().UnixNano(), nil
|
||||
default:
|
||||
return 0, fmt.Errorf("unknown timestamp_precision %q", s.TimeStampUnit)
|
||||
}
|
||||
}
|
||||
|
||||
// convert Metrics to Records with tags
|
||||
func (s *IoTDB) convertMetricsToRecordsWithTags(metrics []telegraf.Metric) (*recordsWithTags, error) {
|
||||
timestampList := make([]int64, 0, len(metrics))
|
||||
deviceidList := make([]string, 0, len(metrics))
|
||||
measurementsList := make([][]string, 0, len(metrics))
|
||||
valuesList := make([][]interface{}, 0, len(metrics))
|
||||
dataTypesList := make([][]client.TSDataType, 0, len(metrics))
|
||||
tagsList := make([][]*telegraf.Tag, 0, len(metrics))
|
||||
|
||||
for _, metric := range metrics {
|
||||
// write `metric` to the output sink here
|
||||
var tags []*telegraf.Tag
|
||||
tags = append(tags, metric.TagList()...)
|
||||
// deal with basic parameter
|
||||
var keys []string
|
||||
var values []interface{}
|
||||
var dataTypes []client.TSDataType
|
||||
for _, field := range metric.FieldList() {
|
||||
datatype, value := s.getDataTypeAndValue(field.Value)
|
||||
if datatype == client.UNKNOWN {
|
||||
return nil, fmt.Errorf("datatype of %q is unknown, values: %v", field.Key, field.Value)
|
||||
}
|
||||
keys = append(keys, field.Key)
|
||||
values = append(values, value)
|
||||
dataTypes = append(dataTypes, datatype)
|
||||
}
|
||||
// Convert timestamp into specified unit
|
||||
ts, err := s.convertTimestampOfMetric(metric)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
timestampList = append(timestampList, ts)
|
||||
// append all metric data of this record to lists
|
||||
deviceidList = append(deviceidList, metric.Name())
|
||||
measurementsList = append(measurementsList, keys)
|
||||
valuesList = append(valuesList, values)
|
||||
dataTypesList = append(dataTypesList, dataTypes)
|
||||
tagsList = append(tagsList, tags)
|
||||
}
|
||||
rwt := &recordsWithTags{
|
||||
DeviceIDList: deviceidList,
|
||||
MeasurementsList: measurementsList,
|
||||
ValuesList: valuesList,
|
||||
DataTypesList: dataTypesList,
|
||||
TimestampList: timestampList,
|
||||
TagsList: tagsList,
|
||||
}
|
||||
return rwt, nil
|
||||
}
|
||||
|
||||
// checks is the tag contains any IoTDB invalid character
|
||||
func (s *IoTDB) validateTag(tag string) (string, error) {
|
||||
// IoTDB uses "root" as a keyword and can be called only at the start of the path
|
||||
if tag == "root" {
|
||||
return "", errors.New("cannot use 'root' as tag")
|
||||
} else if forbiddenBacktick.MatchString(tag) { // returns an error if the backsticks are used in an inappropriate way
|
||||
return "", errors.New("cannot use ` in tag names")
|
||||
} else if allowedBacktick.MatchString(tag) { // if the tag in already enclosed in tags returns the tag
|
||||
return tag, nil
|
||||
}
|
||||
|
||||
// loops through all the regex patterns and if one
|
||||
// pattern matches returns the tag between `
|
||||
for _, regex := range s.sanityRegex {
|
||||
if regex.MatchString(tag) {
|
||||
return "`" + tag + "`", nil
|
||||
}
|
||||
}
|
||||
|
||||
return tag, nil
|
||||
}
|
||||
|
||||
// modify recordsWithTags according to 'TreatTagsAs' Configuration
|
||||
func (s *IoTDB) modifyRecordsWithTags(rwt *recordsWithTags) error {
|
||||
switch s.TreatTagsAs {
|
||||
case "fields":
|
||||
// method 1: treat Tag(Key:Value) as measurement
|
||||
for index, tags := range rwt.TagsList { // for each record
|
||||
for _, tag := range tags { // for each tag of this record, append it's Key:Value to measurements
|
||||
datatype, value := s.getDataTypeAndValue(tag.Value)
|
||||
if datatype == client.UNKNOWN {
|
||||
return fmt.Errorf("datatype of %q is unknown, values: %v", tag.Key, value)
|
||||
}
|
||||
rwt.MeasurementsList[index] = append(rwt.MeasurementsList[index], tag.Key)
|
||||
rwt.ValuesList[index] = append(rwt.ValuesList[index], value)
|
||||
rwt.DataTypesList[index] = append(rwt.DataTypesList[index], datatype)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
case "device_id":
|
||||
// method 2: treat Tag(Key:Value) as subtree of device id
|
||||
for index, tags := range rwt.TagsList { // for each record
|
||||
topic := []string{rwt.DeviceIDList[index]}
|
||||
for _, tag := range tags { // for each tag, append it's Value
|
||||
tagValue, err := s.validateTag(tag.Value) // validates tag
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
topic = append(topic, tagValue)
|
||||
}
|
||||
rwt.DeviceIDList[index] = strings.Join(topic, ".")
|
||||
}
|
||||
return nil
|
||||
default:
|
||||
// something go wrong. This configuration should have been checked in func Init().
|
||||
return fmt.Errorf("unknown 'convert_tags_to' method: %q", s.TreatTagsAs)
|
||||
}
|
||||
}
|
||||
|
||||
// Write records with tags to IoTDB server
|
||||
func (s *IoTDB) writeRecordsWithTags(rwt *recordsWithTags) error {
|
||||
// deal with tags
|
||||
if err := s.modifyRecordsWithTags(rwt); err != nil {
|
||||
return err
|
||||
}
|
||||
// write to IoTDB server
|
||||
status, err := s.session.InsertRecords(rwt.DeviceIDList, rwt.MeasurementsList,
|
||||
rwt.DataTypesList, rwt.ValuesList, rwt.TimestampList)
|
||||
if status != nil {
|
||||
if verifyResult := client.VerifySuccess(status); verifyResult != nil {
|
||||
s.Log.Debug(verifyResult)
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func init() {
|
||||
outputs.Add("iotdb", func() telegraf.Output { return newIoTDB() })
|
||||
}
|
||||
|
||||
// create a new IoTDB struct with default values.
|
||||
func newIoTDB() *IoTDB {
|
||||
return &IoTDB{
|
||||
Host: "localhost",
|
||||
Port: "6667",
|
||||
Timeout: config.Duration(time.Second * 5),
|
||||
ConvertUint64To: "int64_clip",
|
||||
TimeStampUnit: "nanosecond",
|
||||
TreatTagsAs: "device_id",
|
||||
}
|
||||
}
|
641
plugins/outputs/iotdb/iotdb_test.go
Normal file
641
plugins/outputs/iotdb/iotdb_test.go
Normal file
|
@ -0,0 +1,641 @@
|
|||
package iotdb
|
||||
|
||||
import (
|
||||
"math"
|
||||
"strconv"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/apache/iotdb-client-go/client"
|
||||
"github.com/docker/go-connections/nat"
|
||||
"github.com/stretchr/testify/require"
|
||||
"github.com/testcontainers/testcontainers-go/wait"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/config"
|
||||
"github.com/influxdata/telegraf/metric"
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
)
|
||||
|
||||
// newMetricWithOrderedFields creates new Metric and makes sure fields are in
|
||||
// order. This is required to define the expected output where the field order
|
||||
// needs to be defines.
|
||||
func newMetricWithOrderedFields(
|
||||
name string,
|
||||
tags []telegraf.Tag,
|
||||
fields []telegraf.Field,
|
||||
timestamp time.Time,
|
||||
) telegraf.Metric {
|
||||
m := metric.New(name, map[string]string{}, map[string]interface{}{}, timestamp)
|
||||
for _, tag := range tags {
|
||||
m.AddTag(tag.Key, tag.Value)
|
||||
}
|
||||
for _, field := range fields {
|
||||
m.AddField(field.Key, field.Value)
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func TestInitInvalid(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
plugin *IoTDB
|
||||
expected string
|
||||
}{
|
||||
{
|
||||
name: "empty tag-conversion",
|
||||
plugin: func() *IoTDB {
|
||||
s := newIoTDB()
|
||||
s.TreatTagsAs = ""
|
||||
s.Log = &testutil.Logger{}
|
||||
return s
|
||||
}(),
|
||||
expected: `unknown 'convert_tags_to' method ""`,
|
||||
},
|
||||
{
|
||||
name: "empty uint-conversion",
|
||||
plugin: func() *IoTDB {
|
||||
s := newIoTDB()
|
||||
s.ConvertUint64To = ""
|
||||
s.Log = &testutil.Logger{}
|
||||
return s
|
||||
}(),
|
||||
expected: `unknown 'uint64_conversion' method ""`,
|
||||
},
|
||||
{
|
||||
name: "empty timestamp precision",
|
||||
plugin: func() *IoTDB {
|
||||
s := newIoTDB()
|
||||
s.TimeStampUnit = ""
|
||||
s.Log = &testutil.Logger{}
|
||||
return s
|
||||
}(),
|
||||
expected: `unknown 'timestamp_precision' method ""`,
|
||||
},
|
||||
{
|
||||
name: "invalid tag-conversion",
|
||||
plugin: func() *IoTDB {
|
||||
s := newIoTDB()
|
||||
s.TreatTagsAs = "garbage"
|
||||
s.Log = &testutil.Logger{}
|
||||
return s
|
||||
}(),
|
||||
expected: `unknown 'convert_tags_to' method "garbage"`,
|
||||
},
|
||||
{
|
||||
name: "invalid uint-conversion",
|
||||
plugin: func() *IoTDB {
|
||||
s := newIoTDB()
|
||||
s.ConvertUint64To = "garbage"
|
||||
s.Log = &testutil.Logger{}
|
||||
return s
|
||||
}(),
|
||||
expected: `unknown 'uint64_conversion' method "garbage"`,
|
||||
},
|
||||
{
|
||||
name: "invalid timestamp precision",
|
||||
plugin: func() *IoTDB {
|
||||
s := newIoTDB()
|
||||
s.TimeStampUnit = "garbage"
|
||||
s.Log = &testutil.Logger{}
|
||||
return s
|
||||
}(),
|
||||
expected: `unknown 'timestamp_precision' method "garbage"`,
|
||||
},
|
||||
{
|
||||
name: "negative timeout",
|
||||
plugin: func() *IoTDB {
|
||||
s := newIoTDB()
|
||||
s.Timeout = config.Duration(time.Second * -5)
|
||||
s.Log = &testutil.Logger{}
|
||||
return s
|
||||
}(),
|
||||
expected: `negative timeout`,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
require.EqualError(t, tt.plugin.Init(), tt.expected)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Test Metric conversion, which means testing function `convertMetricsToRecordsWithTags`
|
||||
func TestMetricConversionToRecordsWithTags(t *testing.T) {
|
||||
var testTimestamp = time.Date(2022, time.July, 20, 12, 25, 33, 44, time.UTC)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
plugin *IoTDB
|
||||
expected recordsWithTags
|
||||
metrics []telegraf.Metric
|
||||
}{
|
||||
{
|
||||
name: "default config",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); return s }(),
|
||||
expected: recordsWithTags{
|
||||
DeviceIDList: []string{"root.computer.fan", "root.computer.fan", "root.computer.keyboard"},
|
||||
MeasurementsList: [][]string{
|
||||
{"temperature", "counter"},
|
||||
{"counter", "temperature"},
|
||||
{"temperature", "counter", "unsigned_big", "string", "bool", "int_text"},
|
||||
},
|
||||
ValuesList: [][]interface{}{
|
||||
{float64(42.55), int64(987654321)},
|
||||
{int64(123456789), float64(56.24)},
|
||||
{float64(30.33), int64(123456789), int64(math.MaxInt64), "Made in China.", bool(false), "123456789011"},
|
||||
},
|
||||
DataTypesList: [][]client.TSDataType{
|
||||
{client.DOUBLE, client.INT64},
|
||||
{client.INT64, client.DOUBLE},
|
||||
{client.DOUBLE, client.INT64, client.INT64, client.TEXT, client.BOOLEAN, client.TEXT},
|
||||
},
|
||||
TimestampList: []int64{testTimestamp.UnixNano(), testTimestamp.UnixNano(), testTimestamp.UnixNano()},
|
||||
},
|
||||
metrics: []telegraf.Metric{
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.fan",
|
||||
[]telegraf.Tag{
|
||||
{Key: "price", Value: "expensive"},
|
||||
{Key: "owner", Value: "cpu"},
|
||||
},
|
||||
[]telegraf.Field{
|
||||
{Key: "temperature", Value: float64(42.55)},
|
||||
{Key: "counter", Value: int64(987654321)},
|
||||
},
|
||||
testTimestamp,
|
||||
),
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.fan",
|
||||
[]telegraf.Tag{ // same keys in different order
|
||||
{Key: "owner", Value: "gpu"},
|
||||
{Key: "price", Value: "cheap"},
|
||||
},
|
||||
[]telegraf.Field{ // same keys in different order
|
||||
{Key: "counter", Value: int64(123456789)},
|
||||
{Key: "temperature", Value: float64(56.24)},
|
||||
},
|
||||
testTimestamp,
|
||||
),
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.keyboard",
|
||||
nil,
|
||||
[]telegraf.Field{
|
||||
{Key: "temperature", Value: float64(30.33)},
|
||||
{Key: "counter", Value: int64(123456789)},
|
||||
{Key: "unsigned_big", Value: uint64(math.MaxInt64 + 1000)},
|
||||
{Key: "string", Value: "Made in China."},
|
||||
{Key: "bool", Value: bool(false)},
|
||||
{Key: "int_text", Value: "123456789011"},
|
||||
},
|
||||
testTimestamp,
|
||||
),
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "unsigned int to text",
|
||||
plugin: func() *IoTDB { cli002 := newIoTDB(); cli002.ConvertUint64To = "text"; return cli002 }(),
|
||||
expected: recordsWithTags{
|
||||
DeviceIDList: []string{"root.computer.uint_to_text"},
|
||||
MeasurementsList: [][]string{{"unsigned_big"}},
|
||||
ValuesList: [][]interface{}{{strconv.FormatUint(uint64(math.MaxInt64+1000), 10)}},
|
||||
DataTypesList: [][]client.TSDataType{{client.TEXT}},
|
||||
TimestampList: []int64{testTimestamp.UnixNano()},
|
||||
},
|
||||
metrics: []telegraf.Metric{
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.uint_to_text",
|
||||
nil,
|
||||
[]telegraf.Field{
|
||||
{Key: "unsigned_big", Value: uint64(math.MaxInt64 + 1000)},
|
||||
},
|
||||
testTimestamp,
|
||||
),
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "unsigned int to int with overflow",
|
||||
plugin: func() *IoTDB { cli002 := newIoTDB(); cli002.ConvertUint64To = "int64"; return cli002 }(),
|
||||
expected: recordsWithTags{
|
||||
DeviceIDList: []string{"root.computer.overflow"},
|
||||
MeasurementsList: [][]string{{"unsigned_big"}},
|
||||
ValuesList: [][]interface{}{{int64(-9223372036854774809)}},
|
||||
DataTypesList: [][]client.TSDataType{{client.INT64}},
|
||||
TimestampList: []int64{testTimestamp.UnixNano()},
|
||||
},
|
||||
metrics: []telegraf.Metric{
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.overflow",
|
||||
nil,
|
||||
[]telegraf.Field{
|
||||
{Key: "unsigned_big", Value: uint64(math.MaxInt64 + 1000)},
|
||||
},
|
||||
testTimestamp,
|
||||
),
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "second timestamp precision",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); s.TimeStampUnit = "second"; return s }(),
|
||||
expected: recordsWithTags{
|
||||
DeviceIDList: []string{"root.computer.second"},
|
||||
MeasurementsList: [][]string{{"unsigned_big"}},
|
||||
ValuesList: [][]interface{}{{int64(math.MaxInt64)}},
|
||||
DataTypesList: [][]client.TSDataType{{client.INT64}},
|
||||
TimestampList: []int64{testTimestamp.Unix()},
|
||||
},
|
||||
metrics: []telegraf.Metric{
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.second",
|
||||
nil,
|
||||
[]telegraf.Field{
|
||||
{Key: "unsigned_big", Value: uint64(math.MaxInt64 + 1000)},
|
||||
},
|
||||
testTimestamp,
|
||||
),
|
||||
},
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
tt.plugin.Log = &testutil.Logger{}
|
||||
require.NoError(t, tt.plugin.Init())
|
||||
actual, err := tt.plugin.convertMetricsToRecordsWithTags(tt.metrics)
|
||||
require.NoError(t, err)
|
||||
// Ignore the tags-list for comparison
|
||||
actual.TagsList = nil
|
||||
expected := tt.expected
|
||||
require.EqualValues(t, &expected, actual)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Test tag sanitize
|
||||
func TestTagSanitization(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
plugin *IoTDB
|
||||
expected []string
|
||||
input []string
|
||||
}{
|
||||
{ // don't sanitize tags containing UnsopportedCharacter on IoTDB V1.3
|
||||
name: "Don't Sanitize Tags",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); s.SanitizeTags = "1.3"; return s }(),
|
||||
expected: []string{"word", "`word`", "word_"},
|
||||
input: []string{"word", "`word`", "word_"},
|
||||
},
|
||||
{ // sanitize tags containing UnsupportedCharacter on IoTDB V1.3 enclosing them in backticks
|
||||
name: "Sanitize Tags",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); s.SanitizeTags = "1.3"; return s }(),
|
||||
expected: []string{"`wo rd`", "`@`", "`$`", "`#`", "`:`", "`{`", "`}`", "`1`", "`1234`"},
|
||||
input: []string{"wo rd", "@", "$", "#", ":", "{", "}", "1", "1234"},
|
||||
},
|
||||
{ // test on forbidden word and forbidden syntax
|
||||
name: "Errors",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); s.SanitizeTags = "1.3"; return s }(),
|
||||
expected: []string{"", ""},
|
||||
input: []string{"root", "wo`rd"},
|
||||
},
|
||||
{
|
||||
name: "Don't Sanitize Tags",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); s.SanitizeTags = "0.13"; return s }(),
|
||||
expected: []string{"word", "`word`", "word_", "@", "$", "#", ":", "{", "}"},
|
||||
input: []string{"word", "`word`", "word_", "@", "$", "#", ":", "{", "}"},
|
||||
},
|
||||
{ // sanitize tags containing UnsupportedCharacter on IoTDB V0.13 enclosing them in backticks
|
||||
name: "Sanitize Tags",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); s.SanitizeTags = "0.13"; return s }(),
|
||||
expected: []string{"`wo rd`", "`\\`"},
|
||||
input: []string{"wo rd", "\\"},
|
||||
},
|
||||
{ // test on forbidden word and forbidden syntax on IoTDB V0.13
|
||||
name: "Errors",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); s.SanitizeTags = "0.13"; return s }(),
|
||||
expected: []string{"", ""},
|
||||
input: []string{"root", "wo`rd"},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
tt.plugin.Log = &testutil.Logger{}
|
||||
require.NoError(t, tt.plugin.Init())
|
||||
|
||||
actuals := make([]string, 0, len(tt.input))
|
||||
for _, input := range tt.input {
|
||||
//nolint:errcheck // error cases handled by expected vs actual comparison
|
||||
actual, _ := tt.plugin.validateTag(input)
|
||||
actuals = append(actuals, actual)
|
||||
}
|
||||
|
||||
require.EqualValues(t, tt.expected, actuals)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Test tags handling, which means testing function `modifyRecordsWithTags`
|
||||
func TestTagsHandling(t *testing.T) {
|
||||
var testTimestamp = time.Date(2022, time.July, 20, 12, 25, 33, 44, time.UTC)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
plugin *IoTDB
|
||||
expected recordsWithTags
|
||||
input recordsWithTags
|
||||
}{
|
||||
{ // treat tags as fields. And input Tags are NOT in order.
|
||||
name: "treat tags as fields",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); s.TreatTagsAs = "fields"; return s }(),
|
||||
expected: recordsWithTags{
|
||||
DeviceIDList: []string{"root.computer.fields"},
|
||||
MeasurementsList: [][]string{{"temperature", "counter", "owner", "price"}},
|
||||
ValuesList: [][]interface{}{
|
||||
{float64(42.55), int64(987654321), "cpu", "expensive"},
|
||||
},
|
||||
DataTypesList: [][]client.TSDataType{
|
||||
{client.DOUBLE, client.INT64, client.TEXT, client.TEXT},
|
||||
},
|
||||
TimestampList: []int64{testTimestamp.UnixNano()},
|
||||
},
|
||||
input: recordsWithTags{
|
||||
DeviceIDList: []string{"root.computer.fields"},
|
||||
MeasurementsList: [][]string{{"temperature", "counter"}},
|
||||
ValuesList: [][]interface{}{
|
||||
{float64(42.55), int64(987654321)},
|
||||
},
|
||||
DataTypesList: [][]client.TSDataType{
|
||||
{client.DOUBLE, client.INT64},
|
||||
},
|
||||
TimestampList: []int64{testTimestamp.UnixNano()},
|
||||
TagsList: [][]*telegraf.Tag{{
|
||||
{Key: "owner", Value: "cpu"},
|
||||
{Key: "price", Value: "expensive"},
|
||||
}},
|
||||
},
|
||||
},
|
||||
{ // treat tags as device IDs. And input Tags are in order.
|
||||
name: "treat tags as device IDs",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); s.TreatTagsAs = "device_id"; return s }(),
|
||||
expected: recordsWithTags{
|
||||
DeviceIDList: []string{"root.computer.deviceID.cpu.expensive"},
|
||||
MeasurementsList: [][]string{{"temperature", "counter"}},
|
||||
ValuesList: [][]interface{}{
|
||||
{float64(42.55), int64(987654321)},
|
||||
},
|
||||
DataTypesList: [][]client.TSDataType{
|
||||
{client.DOUBLE, client.INT64},
|
||||
},
|
||||
TimestampList: []int64{testTimestamp.UnixNano()},
|
||||
},
|
||||
input: recordsWithTags{
|
||||
DeviceIDList: []string{"root.computer.deviceID"},
|
||||
MeasurementsList: [][]string{{"temperature", "counter"}},
|
||||
ValuesList: [][]interface{}{
|
||||
{float64(42.55), int64(987654321)},
|
||||
},
|
||||
DataTypesList: [][]client.TSDataType{
|
||||
{client.DOUBLE, client.INT64},
|
||||
},
|
||||
TimestampList: []int64{testTimestamp.UnixNano()},
|
||||
TagsList: [][]*telegraf.Tag{{
|
||||
{Key: "owner", Value: "cpu"},
|
||||
{Key: "price", Value: "expensive"},
|
||||
}},
|
||||
},
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
input := tt.input
|
||||
tt.plugin.Log = &testutil.Logger{}
|
||||
require.NoError(t, tt.plugin.Init())
|
||||
require.NoError(t, tt.plugin.modifyRecordsWithTags(&input))
|
||||
// Ignore the tags-list for comparison
|
||||
tt.input.TagsList = nil
|
||||
require.EqualValues(t, tt.expected, tt.input)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Test entire Metric conversion, from metrics to records which are ready to insert
|
||||
func TestEntireMetricConversion(t *testing.T) {
|
||||
var testTimestamp = time.Date(2022, time.July, 20, 12, 25, 33, 44, time.UTC)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
plugin *IoTDB
|
||||
expected recordsWithTags
|
||||
metrics []telegraf.Metric
|
||||
requireEqual bool
|
||||
}{
|
||||
{
|
||||
name: "default config",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); return s }(),
|
||||
expected: recordsWithTags{
|
||||
DeviceIDList: []string{"root.computer.screen.high.LED"},
|
||||
MeasurementsList: [][]string{
|
||||
{"temperature", "counter", "unsigned_big", "string", "bool", "int_text"},
|
||||
},
|
||||
ValuesList: [][]interface{}{
|
||||
{float64(30.33), int64(123456789), int64(math.MaxInt64), "Made in China.", bool(false), "123456789011"},
|
||||
},
|
||||
DataTypesList: [][]client.TSDataType{
|
||||
{client.DOUBLE, client.INT64, client.INT64, client.TEXT, client.BOOLEAN, client.TEXT},
|
||||
},
|
||||
TimestampList: []int64{testTimestamp.UnixNano()},
|
||||
},
|
||||
metrics: []telegraf.Metric{
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.screen",
|
||||
[]telegraf.Tag{
|
||||
{Key: "brightness", Value: "high"},
|
||||
{Key: "type", Value: "LED"},
|
||||
},
|
||||
[]telegraf.Field{
|
||||
{Key: "temperature", Value: float64(30.33)},
|
||||
{Key: "counter", Value: int64(123456789)},
|
||||
{Key: "unsigned_big", Value: uint64(math.MaxInt64 + 1000)},
|
||||
{Key: "string", Value: "Made in China."},
|
||||
{Key: "bool", Value: bool(false)},
|
||||
{Key: "int_text", Value: "123456789011"},
|
||||
},
|
||||
testTimestamp,
|
||||
),
|
||||
},
|
||||
requireEqual: true,
|
||||
},
|
||||
{
|
||||
name: "wrong order of tags",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); return s }(),
|
||||
expected: recordsWithTags{
|
||||
DeviceIDList: []string{"root.computer.screen.LED.high"},
|
||||
MeasurementsList: [][]string{
|
||||
{"temperature", "counter", "unsigned_big", "string", "bool", "int_text"},
|
||||
},
|
||||
ValuesList: [][]interface{}{
|
||||
{float64(30.33), int64(123456789), int64(math.MaxInt64), "Made in China.", bool(false), "123456789011"},
|
||||
},
|
||||
DataTypesList: [][]client.TSDataType{
|
||||
{client.DOUBLE, client.INT64, client.INT64, client.TEXT, client.BOOLEAN, client.TEXT},
|
||||
},
|
||||
TimestampList: []int64{testTimestamp.UnixNano()},
|
||||
},
|
||||
metrics: []telegraf.Metric{
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.screen",
|
||||
[]telegraf.Tag{
|
||||
{Key: "brightness", Value: "high"},
|
||||
{Key: "type", Value: "LED"},
|
||||
},
|
||||
[]telegraf.Field{
|
||||
{Key: "temperature", Value: float64(30.33)},
|
||||
{Key: "counter", Value: int64(123456789)},
|
||||
{Key: "unsigned_big", Value: uint64(math.MaxInt64 + 1000)},
|
||||
{Key: "string", Value: "Made in China."},
|
||||
{Key: "bool", Value: bool(false)},
|
||||
{Key: "int_text", Value: "123456789011"},
|
||||
},
|
||||
testTimestamp,
|
||||
),
|
||||
},
|
||||
requireEqual: false,
|
||||
},
|
||||
{
|
||||
name: "wrong order of tags",
|
||||
plugin: func() *IoTDB { s := newIoTDB(); return s }(),
|
||||
expected: recordsWithTags{
|
||||
DeviceIDList: []string{"root.computer.screen.LED.high"},
|
||||
MeasurementsList: [][]string{
|
||||
{"temperature", "counter"},
|
||||
},
|
||||
ValuesList: [][]interface{}{
|
||||
{float64(30.33), int64(123456789)},
|
||||
},
|
||||
DataTypesList: [][]client.TSDataType{
|
||||
{client.DOUBLE, client.INT64},
|
||||
},
|
||||
TimestampList: []int64{testTimestamp.UnixNano()},
|
||||
},
|
||||
metrics: []telegraf.Metric{
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.screen",
|
||||
[]telegraf.Tag{
|
||||
{Key: "brightness", Value: "high"},
|
||||
{Key: "type", Value: "LED"},
|
||||
},
|
||||
[]telegraf.Field{
|
||||
{Key: "temperature", Value: float64(30.33)},
|
||||
{Key: "counter", Value: int64(123456789)},
|
||||
},
|
||||
testTimestamp,
|
||||
),
|
||||
},
|
||||
requireEqual: false,
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
tt.plugin.Log = &testutil.Logger{}
|
||||
require.NoError(t, tt.plugin.Init())
|
||||
actual, err := tt.plugin.convertMetricsToRecordsWithTags(tt.metrics)
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, tt.plugin.modifyRecordsWithTags(actual))
|
||||
// Ignore the tags-list for comparison
|
||||
actual.TagsList = nil
|
||||
expected := tt.expected
|
||||
if tt.requireEqual {
|
||||
require.EqualValues(t, &expected, actual)
|
||||
} else {
|
||||
require.NotEqualValues(t, &expected, actual)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Start a container and do integration test.
|
||||
func TestIntegrationInserts(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("Skipping integration test in short mode")
|
||||
}
|
||||
const iotdbPort = "6667"
|
||||
|
||||
container := testutil.Container{
|
||||
Image: "apache/iotdb:0.13.0-node",
|
||||
ExposedPorts: []string{iotdbPort},
|
||||
WaitingFor: wait.ForAll(
|
||||
wait.ForListeningPort(nat.Port(iotdbPort)),
|
||||
wait.ForLog("IoTDB has started."),
|
||||
),
|
||||
}
|
||||
err := container.Start()
|
||||
require.NoError(t, err, "failed to start IoTDB container")
|
||||
defer container.Terminate()
|
||||
|
||||
t.Logf("Container Address:%q, ExposedPorts:[%v:%v]", container.Address, container.Ports[iotdbPort], iotdbPort)
|
||||
// create a client and tests two groups of insertion
|
||||
testClient := &IoTDB{
|
||||
Host: container.Address,
|
||||
Port: container.Ports[iotdbPort],
|
||||
User: config.NewSecret([]byte("root")),
|
||||
Password: config.NewSecret([]byte("root")),
|
||||
Timeout: config.Duration(time.Second * 5),
|
||||
ConvertUint64To: "int64_clip",
|
||||
TimeStampUnit: "nanosecond",
|
||||
TreatTagsAs: "device_id",
|
||||
}
|
||||
testClient.Log = &testutil.Logger{}
|
||||
|
||||
// generate Metrics to input
|
||||
metrics := []telegraf.Metric{
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.unsigned_big",
|
||||
nil,
|
||||
[]telegraf.Field{
|
||||
{Key: "unsigned_big", Value: uint64(math.MaxInt64 + 1000)},
|
||||
},
|
||||
time.Now(),
|
||||
),
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.fan",
|
||||
[]telegraf.Tag{
|
||||
{Key: "price", Value: "expensive"},
|
||||
{Key: "owner", Value: "cpu"},
|
||||
},
|
||||
[]telegraf.Field{
|
||||
{Key: "temperature", Value: float64(42.55)},
|
||||
{Key: "counter", Value: int64(987654321)},
|
||||
},
|
||||
time.Now(),
|
||||
),
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.fan",
|
||||
[]telegraf.Tag{ // same keys in different order
|
||||
{Key: "owner", Value: "gpu"},
|
||||
{Key: "price", Value: "cheap"},
|
||||
},
|
||||
[]telegraf.Field{ // same keys in different order
|
||||
{Key: "counter", Value: int64(123456789)},
|
||||
{Key: "temperature", Value: float64(56.24)},
|
||||
},
|
||||
time.Now(),
|
||||
),
|
||||
newMetricWithOrderedFields(
|
||||
"root.computer.keyboard",
|
||||
nil,
|
||||
[]telegraf.Field{
|
||||
{Key: "temperature", Value: float64(30.33)},
|
||||
{Key: "counter", Value: int64(123456789)},
|
||||
{Key: "unsigned_big", Value: uint64(math.MaxInt64 + 1000)},
|
||||
{Key: "string", Value: "Made in China."},
|
||||
{Key: "bool", Value: bool(false)},
|
||||
{Key: "int_text", Value: "123456789011"},
|
||||
},
|
||||
time.Now(),
|
||||
),
|
||||
}
|
||||
|
||||
require.NoError(t, testClient.Connect())
|
||||
require.NoError(t, testClient.Write(metrics))
|
||||
require.NoError(t, testClient.Close())
|
||||
}
|
59
plugins/outputs/iotdb/sample.conf
Normal file
59
plugins/outputs/iotdb/sample.conf
Normal file
|
@ -0,0 +1,59 @@
|
|||
# Save metrics to an IoTDB Database
|
||||
[[outputs.iotdb]]
|
||||
## Configuration of IoTDB server connection
|
||||
host = "127.0.0.1"
|
||||
# port = "6667"
|
||||
|
||||
## Configuration of authentication
|
||||
# user = "root"
|
||||
# password = "root"
|
||||
|
||||
## Timeout to open a new session.
|
||||
## A value of zero means no timeout.
|
||||
# timeout = "5s"
|
||||
|
||||
## Configuration of type conversion for 64-bit unsigned int
|
||||
## IoTDB currently DOES NOT support unsigned integers (version 13.x).
|
||||
## 32-bit unsigned integers are safely converted into 64-bit signed integers by the plugin,
|
||||
## however, this is not true for 64-bit values in general as overflows may occur.
|
||||
## The following setting allows to specify the handling of 64-bit unsigned integers.
|
||||
## Available values are:
|
||||
## - "int64" -- convert to 64-bit signed integers and accept overflows
|
||||
## - "int64_clip" -- convert to 64-bit signed integers and clip the values on overflow to 9,223,372,036,854,775,807
|
||||
## - "text" -- convert to the string representation of the value
|
||||
# uint64_conversion = "int64_clip"
|
||||
|
||||
## Configuration of TimeStamp
|
||||
## TimeStamp is always saved in 64bits int. timestamp_precision specifies the unit of timestamp.
|
||||
## Available value:
|
||||
## "second", "millisecond", "microsecond", "nanosecond"(default)
|
||||
# timestamp_precision = "nanosecond"
|
||||
|
||||
## Handling of tags
|
||||
## Tags are not fully supported by IoTDB.
|
||||
## A guide with suggestions on how to handle tags can be found here:
|
||||
## https://iotdb.apache.org/UserGuide/Master/API/InfluxDB-Protocol.html
|
||||
##
|
||||
## Available values are:
|
||||
## - "fields" -- convert tags to fields in the measurement
|
||||
## - "device_id" -- attach tags to the device ID
|
||||
##
|
||||
## For Example, a metric named "root.sg.device" with the tags `tag1: "private"` and `tag2: "working"` and
|
||||
## fields `s1: 100` and `s2: "hello"` will result in the following representations in IoTDB
|
||||
## - "fields" -- root.sg.device, s1=100, s2="hello", tag1="private", tag2="working"
|
||||
## - "device_id" -- root.sg.device.private.working, s1=100, s2="hello"
|
||||
# convert_tags_to = "device_id"
|
||||
|
||||
## Handling of unsupported characters
|
||||
## Some characters in different versions of IoTDB are not supported in path name
|
||||
## A guide with suggetions on valid paths can be found here:
|
||||
## for iotdb 0.13.x -> https://iotdb.apache.org/UserGuide/V0.13.x/Reference/Syntax-Conventions.html#identifiers
|
||||
## for iotdb 1.x.x and above -> https://iotdb.apache.org/UserGuide/V1.3.x/User-Manual/Syntax-Rule.html#identifier
|
||||
##
|
||||
## Available values are:
|
||||
## - "1.0", "1.1", "1.2", "1.3" -- use backticks to enclose tags with forbidden characters
|
||||
## such as @$#:[]{}() and space
|
||||
## - "0.13" -- use backticks to enclose tags with forbidden characters
|
||||
## such as space
|
||||
## Keep this section commented if you don't want to sanitize the path
|
||||
# sanitize_tag = "1.3"
|
Loading…
Add table
Add a link
Reference in a new issue