Adding upstream version 1.34.4.
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
parent
e393c3af3f
commit
4978089aab
4963 changed files with 677545 additions and 0 deletions
186
plugins/processors/strings/README.md
Normal file
186
plugins/processors/strings/README.md
Normal file
|
@ -0,0 +1,186 @@
|
|||
# Strings Processor Plugin
|
||||
|
||||
The `strings` plugin maps certain go string functions onto measurement, tag, and
|
||||
field values. Values can be modified in place or stored in another key.
|
||||
|
||||
Implemented functions are:
|
||||
|
||||
- lowercase
|
||||
- uppercase
|
||||
- titlecase
|
||||
- trim
|
||||
- trim_left
|
||||
- trim_right
|
||||
- trim_prefix
|
||||
- trim_suffix
|
||||
- replace
|
||||
- left
|
||||
- base64decode
|
||||
- valid_utf8
|
||||
|
||||
Please note that in this implementation these are processed in the order that
|
||||
they appear above.
|
||||
|
||||
Specify the `measurement`, `tag`, `tag_key`, `field`, or `field_key` that you
|
||||
want processed in each section and optionally a `dest` if you want the result
|
||||
stored in a new tag or field. You can specify lots of transformations on data
|
||||
with a single strings processor.
|
||||
|
||||
If you'd like to apply the change to every `tag`, `tag_key`, `field`,
|
||||
`field_key`, or `measurement`, use the value `"*"` for each respective
|
||||
field. Note that the `dest` field will be ignored if `"*"` is used.
|
||||
|
||||
If you'd like to apply multiple processings to the same `tag_key` or
|
||||
`field_key`, note the process order stated above. See the second example below
|
||||
for an example.
|
||||
|
||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||
|
||||
In addition to the plugin-specific configuration settings, plugins support
|
||||
additional global and plugin configuration settings. These settings are used to
|
||||
modify metrics, tags, and field or create aliases and configure ordering, etc.
|
||||
See the [CONFIGURATION.md][CONFIGURATION.md] for more details.
|
||||
|
||||
[CONFIGURATION.md]: ../../../docs/CONFIGURATION.md#plugins
|
||||
|
||||
## Configuration
|
||||
|
||||
```toml @sample.conf
|
||||
# Perform string processing on tags, fields, and measurements
|
||||
[[processors.strings]]
|
||||
## Convert a field value to lowercase and store in a new field
|
||||
# [[processors.strings.lowercase]]
|
||||
# field = "uri_stem"
|
||||
# dest = "uri_stem_normalised"
|
||||
|
||||
## Convert a tag value to uppercase
|
||||
# [[processors.strings.uppercase]]
|
||||
# tag = "method"
|
||||
|
||||
## Convert a field value to titlecase
|
||||
# [[processors.strings.titlecase]]
|
||||
# field = "status"
|
||||
|
||||
## Trim leading and trailing whitespace using the default cutset
|
||||
# [[processors.strings.trim]]
|
||||
# field = "message"
|
||||
|
||||
## Trim leading characters in cutset
|
||||
# [[processors.strings.trim_left]]
|
||||
# field = "message"
|
||||
# cutset = "\t"
|
||||
|
||||
## Trim trailing characters in cutset
|
||||
# [[processors.strings.trim_right]]
|
||||
# field = "message"
|
||||
# cutset = "\r\n"
|
||||
|
||||
## Trim the given prefix from the field
|
||||
# [[processors.strings.trim_prefix]]
|
||||
# field = "my_value"
|
||||
# prefix = "my_"
|
||||
|
||||
## Trim the given suffix from the field
|
||||
# [[processors.strings.trim_suffix]]
|
||||
# field = "read_count"
|
||||
# suffix = "_count"
|
||||
|
||||
## Replace all non-overlapping instances of old with new
|
||||
# [[processors.strings.replace]]
|
||||
# measurement = "*"
|
||||
# old = ":"
|
||||
# new = "_"
|
||||
|
||||
## Trims strings based on width
|
||||
# [[processors.strings.left]]
|
||||
# field = "message"
|
||||
# width = 10
|
||||
|
||||
## Decode a base64 encoded utf-8 string
|
||||
# [[processors.strings.base64decode]]
|
||||
# field = "message"
|
||||
|
||||
## Sanitize a string to ensure it is a valid utf-8 string
|
||||
## Each run of invalid UTF-8 byte sequences is replaced by the replacement string, which may be empty
|
||||
# [[processors.strings.valid_utf8]]
|
||||
# field = "message"
|
||||
# replacement = ""
|
||||
```
|
||||
|
||||
### Trim, TrimLeft, TrimRight
|
||||
|
||||
The `trim`, `trim_left`, and `trim_right` functions take an optional parameter:
|
||||
`cutset`. This value is a string containing the characters to remove from the
|
||||
value.
|
||||
|
||||
### TrimPrefix, TrimSuffix
|
||||
|
||||
The `trim_prefix` and `trim_suffix` functions remote the given `prefix` or
|
||||
`suffix` respectively from the string.
|
||||
|
||||
### Replace
|
||||
|
||||
The `replace` function does a substring replacement across the entire
|
||||
string to allow for different conventions between various input and output
|
||||
plugins. Some example usages are eliminating disallowed characters in
|
||||
field names or replacing separators between different separators.
|
||||
Can also be used to eliminate unneeded chars that were in metrics.
|
||||
If the entire name would be deleted, it will refuse to perform
|
||||
the operation and keep the old name.
|
||||
|
||||
## Example
|
||||
|
||||
A sample configuration:
|
||||
|
||||
```toml
|
||||
[[processors.strings]]
|
||||
[[processors.strings.lowercase]]
|
||||
tag = "uri_stem"
|
||||
|
||||
[[processors.strings.trim_prefix]]
|
||||
tag = "uri_stem"
|
||||
prefix = "/api/"
|
||||
|
||||
[[processors.strings.uppercase]]
|
||||
field = "cs-host"
|
||||
dest = "cs-host_normalised"
|
||||
```
|
||||
|
||||
Sample input:
|
||||
|
||||
```text
|
||||
iis_log,method=get,uri_stem=/API/HealthCheck cs-host="MIXEDCASE_host",http_version=1.1 1519652321000000000
|
||||
```
|
||||
|
||||
Sample output:
|
||||
|
||||
```text
|
||||
iis_log,method=get,uri_stem=healthcheck cs-host="MIXEDCASE_host",http_version=1.1,cs-host_normalised="MIXEDCASE_HOST" 1519652321000000000
|
||||
```
|
||||
|
||||
### Second Example
|
||||
|
||||
A sample configuration:
|
||||
|
||||
```toml
|
||||
[[processors.strings]]
|
||||
[[processors.strings.lowercase]]
|
||||
tag_key = "URI-Stem"
|
||||
|
||||
[[processors.strings.replace]]
|
||||
tag_key = "uri-stem"
|
||||
old = "-"
|
||||
new = "_"
|
||||
```
|
||||
|
||||
Sample input:
|
||||
|
||||
```text
|
||||
iis_log,URI-Stem=/API/HealthCheck http_version=1.1 1519652321000000000
|
||||
```
|
||||
|
||||
Sample output:
|
||||
|
||||
```text
|
||||
iis_log,uri_stem=/API/HealthCheck http_version=1.1 1519652321000000000
|
||||
```
|
Loading…
Add table
Add a link
Reference in a new issue