telegraf/plugins/processors/strings/README.md

# Strings Processor Plugin

The `strings` plugin maps certain go string functions onto measurement, tag, and
field values.  Values can be modified in place or stored in another key.

Implemented functions are:

- lowercase
- uppercase
- titlecase
- trim
- trim_left
- trim_right
- trim_prefix
- trim_suffix
- replace
- left
- base64decode
- valid_utf8

Please note that in this implementation these are processed in the order that
they appear above.

Specify the `measurement`, `tag`, `tag_key`, `field`, or `field_key` that you
want processed in each section and optionally a `dest` if you want the result
stored in a new tag or field. You can specify lots of transformations on data
with a single strings processor.

If you'd like to apply the change to every `tag`, `tag_key`, `field`,
`field_key`, or `measurement`, use the value `"*"` for each respective
field. Note that the `dest` field will be ignored if `"*"` is used.

If you'd like to apply multiple processings to the same `tag_key` or
`field_key`, note the process order stated above. See the second example below
for an example.

## Global configuration options <!-- @/docs/includes/plugin_config.md -->

In addition to the plugin-specific configuration settings, plugins support
additional global and plugin configuration settings. These settings are used to
modify metrics, tags, and field or create aliases and configure ordering, etc.
See the [CONFIGURATION.md][CONFIGURATION.md] for more details.

[CONFIGURATION.md]: ../../../docs/CONFIGURATION.md#plugins

## Configuration

```toml @sample.conf
# Perform string processing on tags, fields, and measurements
[[processors.strings]]
  ## Convert a field value to lowercase and store in a new field
  # [[processors.strings.lowercase]]
  #   field = "uri_stem"
  #   dest = "uri_stem_normalised"

  ## Convert a tag value to uppercase
  # [[processors.strings.uppercase]]
  #   tag = "method"

  ## Convert a field value to titlecase
  # [[processors.strings.titlecase]]
  #   field = "status"

  ## Trim leading and trailing whitespace using the default cutset
  # [[processors.strings.trim]]
  #   field = "message"

  ## Trim leading characters in cutset
  # [[processors.strings.trim_left]]
  #   field = "message"
  #   cutset = "\t"

  ## Trim trailing characters in cutset
  # [[processors.strings.trim_right]]
  #   field = "message"
  #   cutset = "\r\n"

  ## Trim the given prefix from the field
  # [[processors.strings.trim_prefix]]
  #   field = "my_value"
  #   prefix = "my_"

  ## Trim the given suffix from the field
  # [[processors.strings.trim_suffix]]
  #   field = "read_count"
  #   suffix = "_count"

  ## Replace all non-overlapping instances of old with new
  # [[processors.strings.replace]]
  #   measurement = "*"
  #   old = ":"
  #   new = "_"

  ## Trims strings based on width
  # [[processors.strings.left]]
  #   field = "message"
  #   width = 10

  ## Decode a base64 encoded utf-8 string
  # [[processors.strings.base64decode]]
  #   field = "message"

  ## Sanitize a string to ensure it is a valid utf-8 string
  ## Each run of invalid UTF-8 byte sequences is replaced by the replacement string, which may be empty
  # [[processors.strings.valid_utf8]]
  #   field = "message"
  #   replacement = ""
```

### Trim, TrimLeft, TrimRight

The `trim`, `trim_left`, and `trim_right` functions take an optional parameter:
`cutset`.  This value is a string containing the characters to remove from the
value.

### TrimPrefix, TrimSuffix

The `trim_prefix` and `trim_suffix` functions remote the given `prefix` or
`suffix` respectively from the string.

### Replace

The `replace` function does a substring replacement across the entire
string to allow for different conventions between various input and output
plugins. Some example usages are eliminating disallowed characters in
field names or replacing separators between different separators.
Can also be used to eliminate unneeded chars that were in metrics.
If the entire name would be deleted, it will refuse to perform
the operation and keep the old name.

## Example

A sample configuration:

```toml
[[processors.strings]]
  [[processors.strings.lowercase]]
    tag = "uri_stem"

  [[processors.strings.trim_prefix]]
    tag = "uri_stem"
    prefix = "/api/"

  [[processors.strings.uppercase]]
    field = "cs-host"
    dest = "cs-host_normalised"
```

Sample input:

```text
iis_log,method=get,uri_stem=/API/HealthCheck cs-host="MIXEDCASE_host",http_version=1.1 1519652321000000000
```

Sample output:

```text
iis_log,method=get,uri_stem=healthcheck cs-host="MIXEDCASE_host",http_version=1.1,cs-host_normalised="MIXEDCASE_HOST" 1519652321000000000
```

### Second Example

A sample configuration:

```toml
[[processors.strings]]
  [[processors.strings.lowercase]]
    tag_key = "URI-Stem"

  [[processors.strings.replace]]
    tag_key = "uri-stem"
    old = "-"
    new = "_"
```

Sample input:

```text
iis_log,URI-Stem=/API/HealthCheck http_version=1.1 1519652321000000000
```

Sample output:

```text
iis_log,uri_stem=/API/HealthCheck http_version=1.1 1519652321000000000
```