influx/telegraf
1
0
Fork 0
Code Issues Activity

Adding upstream version 1.34.4.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-05-24 07:26:29 +02:00
parent e393c3af3f
commit 4978089aab
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
4963 changed files with 677545 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

166
plugins/inputs/http/README.md Normal file
View file

@ -0,0 +1,166 @@
# HTTP Input Plugin
This plugin collects metrics from one or more HTTP endpoints providing data in
one of the supported [data formats][data_formats].
⭐ Telegraf v1.6.0
🏷️ applications, server
💻 all
[data_formats]: /docs/DATA_FORMATS_INPUT.md
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
In addition to the plugin-specific configuration settings, plugins support
additional global and plugin configuration settings. These settings are used to
modify metrics, tags, and field or create aliases and configure ordering, etc.
See the [CONFIGURATION.md][CONFIGURATION.md] for more details.
[CONFIGURATION.md]: ../../../docs/CONFIGURATION.md#plugins
## Secret-store support
This plugin supports secrets from secret-stores for the `username`, `password`,
`token`, `headers`, and `cookie_auth_headers` option.
See the [secret-store documentation][SECRETSTORE] for more details on how
to use them.
[SECRETSTORE]: ../../../docs/CONFIGURATION.md#secret-store-secrets
## Configuration
```toml @sample.conf
# Read formatted metrics from one or more HTTP endpoints
[[inputs.http]]
## One or more URLs from which to read formatted metrics.
urls = [
"http://localhost/metrics",
"http+unix:///run/user/420/podman/podman.sock:/d/v4.0.0/libpod/pods/json"
]
## HTTP method
# method = "GET"
## Optional HTTP headers
# headers = {"X-Special-Header" = "Special-Value"}
## HTTP entity-body to send with POST/PUT requests.
# body = ""
## HTTP Content-Encoding for write request body, can be set to "gzip" to
## compress body or "identity" to apply no encoding.
# content_encoding = "identity"
## Optional Bearer token settings to use for the API calls.
## Use either the token itself or the token file if you need a token.
# token = "eyJhbGc...Qssw5c"
# token_file = "/path/to/file"
## Optional HTTP Basic Auth Credentials
# username = "username"
# password = "pa$$word"
## OAuth2 Client Credentials. The options 'client_id', 'client_secret', and 'token_url' are required to use OAuth2.
# client_id = "clientid"
# client_secret = "secret"
# token_url = "https://indentityprovider/oauth2/v1/token"
# scopes = ["urn:opc:idm:__myscopes__"]
## HTTP Proxy support
# use_system_proxy = false
# http_proxy_url = ""
## Optional TLS Config
## Set to true/false to enforce TLS being enabled/disabled. If not set,
## enable TLS only if any of the other options are specified.
# tls_enable =
## Trusted root certificates for server
# tls_ca = "/path/to/cafile"
## Used for TLS client certificate authentication
# tls_cert = "/path/to/certfile"
## Used for TLS client certificate authentication
# tls_key = "/path/to/keyfile"
## Password for the key file if it is encrypted
# tls_key_pwd = ""
## Send the specified TLS server name via SNI
# tls_server_name = "kubernetes.example.com"
## Minimal TLS version to accept by the client
# tls_min_version = "TLS12"
## List of ciphers to accept, by default all secure ciphers will be accepted
## See https://pkg.go.dev/crypto/tls#pkg-constants for supported values.
## Use "all", "secure" and "insecure" to add all support ciphers, secure
## suites or insecure suites respectively.
# tls_cipher_suites = ["secure"]
## Renegotiation method, "never", "once" or "freely"
# tls_renegotiation_method = "never"
## Use TLS but skip chain & host verification
# insecure_skip_verify = false
## Optional Cookie authentication
# cookie_auth_url = "https://localhost/authMe"
# cookie_auth_method = "POST"
# cookie_auth_username = "username"
# cookie_auth_password = "pa$$word"
# cookie_auth_headers = { Content-Type = "application/json", X-MY-HEADER = "hello" }
# cookie_auth_body = '{"username": "user", "password": "pa$$word", "authenticate": "me"}'
## cookie_auth_renewal not set or set to "0" will auth once and never renew the cookie
# cookie_auth_renewal = "5m"
## Amount of time allowed to complete the HTTP request
# timeout = "5s"
## List of success status codes
# success_status_codes = [200]
## Data format to consume.
## Each data format has its own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
# data_format = "influx"
```
HTTP requests over Unix domain sockets can be specified via the "http+unix" or
"https+unix" schemes.
Request URLs should have the following form:
```text
http+unix:///path/to/service.sock:/api/endpoint
```
Note: The path to the Unix domain socket and the request endpoint are separated
by a colon (":").
## Example Output
This example output was taken from [this instructional article][1].
[1]: https://docs.influxdata.com/telegraf/v1/configure_plugins/input_plugins/using_http/
```text
citibike,station_id=4703 eightd_has_available_keys=false,is_installed=1,is_renting=1,is_returning=1,legacy_id="4703",num_bikes_available=6,num_bikes_disabled=2,num_docks_available=26,num_docks_disabled=0,num_ebikes_available=0,station_status="active" 1641505084000000000
citibike,station_id=4704 eightd_has_available_keys=false,is_installed=1,is_renting=1,is_returning=1,legacy_id="4704",num_bikes_available=10,num_bikes_disabled=2,num_docks_available=36,num_docks_disabled=0,num_ebikes_available=0,station_status="active" 1641505084000000000
citibike,station_id=4711 eightd_has_available_keys=false,is_installed=1,is_renting=1,is_returning=1,legacy_id="4711",num_bikes_available=9,num_bikes_disabled=0,num_docks_available=36,num_docks_disabled=0,num_ebikes_available=1,station_status="active" 1641505084000000000
```
## Metrics
The metrics collected by this input plugin will depend on the configured
`data_format` and the payload returned by the HTTP endpoint(s).
The default values below are added if the input format does not specify a value:
- http
- tags:
- url
## Optional Cookie Authentication Settings
The optional Cookie Authentication Settings will retrieve a cookie from the
given authorization endpoint, and use it in subsequent API requests. This is
useful for services that do not provide OAuth or Basic Auth authentication,
e.g. the [Tesla Powerwall API][tesla], which uses a Cookie Auth Body to retrieve
an authorization cookie. The Cookie Auth Renewal interval will renew the
authorization by retrieving a new cookie at the given interval.
[tesla]: https://www.tesla.com/support/energy/powerwall/own/monitoring-from-home-network

266
plugins/inputs/http/http.go Normal file
View file

@ -0,0 +1,266 @@
//go:generate ../../../tools/config_includer/generator
//go:generate ../../../tools/readme_config_includer/generator
package http
import (
"context"
_ "embed"
"errors"
"fmt"
"io"
"net/http"
"os"
"strings"
"sync"
"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/config"
"github.com/influxdata/telegraf/internal"
common_http "github.com/influxdata/telegraf/plugins/common/http"
"github.com/influxdata/telegraf/plugins/inputs"
)
//go:embed sample.conf
var sampleConfig string
var once sync.Once
type HTTP struct {
URLs []string `toml:"urls"`
Method string `toml:"method"`
Body string `toml:"body"`
ContentEncoding string `toml:"content_encoding"`
// Basic authentication
Username config.Secret `toml:"username"`
Password config.Secret `toml:"password"`
// Bearer authentication
BearerToken string `toml:"bearer_token" deprecated:"1.28.0;1.35.0;use 'token_file' instead"`
Token config.Secret `toml:"token"`
TokenFile string `toml:"token_file"`
Headers map[string]*config.Secret `toml:"headers"`
SuccessStatusCodes []int `toml:"success_status_codes"`
Log telegraf.Logger `toml:"-"`
common_http.HTTPClientConfig
client *http.Client
parserFunc telegraf.ParserFunc
}
func (*HTTP) SampleConfig() string {
return sampleConfig
}
func (h *HTTP) Init() error {
// For backward compatibility
if h.TokenFile != "" && h.BearerToken != "" && h.TokenFile != h.BearerToken {
return errors.New("conflicting settings for 'bearer_token' and 'token_file'")
} else if h.TokenFile == "" && h.BearerToken != "" {
h.TokenFile = h.BearerToken
}
// We cannot use multiple sources for tokens
if h.TokenFile != "" && !h.Token.Empty() {
return errors.New("either use 'token_file' or 'token' not both")
}
// Create the client
ctx := context.Background()
client, err := h.HTTPClientConfig.CreateClient(ctx, h.Log)
if err != nil {
return err
}
h.client = client
// Set default as [200]
if len(h.SuccessStatusCodes) == 0 {
h.SuccessStatusCodes = []int{200}
}
return nil
}
func (h *HTTP) SetParserFunc(fn telegraf.ParserFunc) {
h.parserFunc = fn
}
func (*HTTP) Start(telegraf.Accumulator) error {
return nil
}
func (h *HTTP) Gather(acc telegraf.Accumulator) error {
var wg sync.WaitGroup
for _, u := range h.URLs {
wg.Add(1)
go func(url string) {
defer wg.Done()
if err := h.gatherURL(acc, url); err != nil {
acc.AddError(fmt.Errorf("[url=%s]: %w", url, err))
}
}(u)
}
wg.Wait()
return nil
}
func (h *HTTP) Stop() {
if h.client != nil {
h.client.CloseIdleConnections()
}
}
// Gathers data from a particular URL
// Parameters:
//
// acc : The telegraf Accumulator to use
// url : endpoint to send request to
//
// Returns:
//
// error: Any error that may have occurred
func (h *HTTP) gatherURL(acc telegraf.Accumulator, url string) error {
body := makeRequestBodyReader(h.ContentEncoding, h.Body)
request, err := http.NewRequest(h.Method, url, body)
if err != nil {
return err
}
if !h.Token.Empty() {
token, err := h.Token.Get()
if err != nil {
return err
}
bearer := "Bearer " + strings.TrimSpace(token.String())
token.Destroy()
request.Header.Set("Authorization", bearer)
} else if h.TokenFile != "" {
token, err := os.ReadFile(h.TokenFile)
if err != nil {
return err
}
bearer := "Bearer " + strings.Trim(string(token), "\n")
request.Header.Set("Authorization", bearer)
}
if h.ContentEncoding == "gzip" {
request.Header.Set("Content-Encoding", "gzip")
}
for k, v := range h.Headers {
secret, err := v.Get()
if err != nil {
return err
}
headerVal := secret.String()
if strings.EqualFold(k, "host") {
request.Host = headerVal
} else {
request.Header.Add(k, headerVal)
}
secret.Destroy()
}
if err := h.setRequestAuth(request); err != nil {
return err
}
resp, err := h.client.Do(request)
if err != nil {
return err
}
defer resp.Body.Close()
responseHasSuccessCode := false
for _, statusCode := range h.SuccessStatusCodes {
if resp.StatusCode == statusCode {
responseHasSuccessCode = true
break
}
}
if !responseHasSuccessCode {
return fmt.Errorf("received status code %d (%s), expected any value out of %v",
resp.StatusCode,
http.StatusText(resp.StatusCode),
h.SuccessStatusCodes)
}
b, err := io.ReadAll(resp.Body)
if err != nil {
return fmt.Errorf("reading body failed: %w", err)
}
// Instantiate a new parser for the new data to avoid trouble with stateful parsers
parser, err := h.parserFunc()
if err != nil {
return fmt.Errorf("instantiating parser failed: %w", err)
}
metrics, err := parser.Parse(b)
if err != nil {
return fmt.Errorf("parsing metrics failed: %w", err)
}
if len(metrics) == 0 {
once.Do(func() {
h.Log.Debug(internal.NoMetricsCreatedMsg)
})
}
for _, metric := range metrics {
if !metric.HasTag("url") {
metric.AddTag("url", url)
}
acc.AddFields(metric.Name(), metric.Fields(), metric.Tags(), metric.Time())
}
return nil
}
func (h *HTTP) setRequestAuth(request *http.Request) error {
if h.Username.Empty() && h.Password.Empty() {
return nil
}
username, err := h.Username.Get()
if err != nil {
return fmt.Errorf("getting username failed: %w", err)
}
defer username.Destroy()
password, err := h.Password.Get()
if err != nil {
return fmt.Errorf("getting password failed: %w", err)
}
defer password.Destroy()
request.SetBasicAuth(username.String(), password.String())
return nil
}
func makeRequestBodyReader(contentEncoding, body string) io.Reader {
if body == "" {
return nil
}
var reader io.Reader = strings.NewReader(body)
if contentEncoding == "gzip" {
return internal.CompressWithGzip(reader)
}
return reader
}
func init() {
inputs.Add("http", func() telegraf.Input {
return &HTTP{
Method: "GET",
}
})
}

549
plugins/inputs/http/http_test.go Normal file
View file

@ -0,0 +1,549 @@
package http_test
import (
"compress/gzip"
"fmt"
"io"
"math/rand"
"net"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/stretchr/testify/require"
"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/config"
common_http "github.com/influxdata/telegraf/plugins/common/http"
"github.com/influxdata/telegraf/plugins/common/oauth"
httpplugin "github.com/influxdata/telegraf/plugins/inputs/http"
"github.com/influxdata/telegraf/plugins/parsers/csv"
"github.com/influxdata/telegraf/plugins/parsers/influx"
"github.com/influxdata/telegraf/plugins/parsers/json"
"github.com/influxdata/telegraf/plugins/parsers/value"
"github.com/influxdata/telegraf/testutil"
)
func TestHTTPWithJSONFormat(t *testing.T) {
fakeServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/endpoint" {
if _, err := w.Write([]byte(simpleJSON)); err != nil {
w.WriteHeader(http.StatusInternalServerError)
t.Error(err)
return
}
} else {
w.WriteHeader(http.StatusNotFound)
}
}))
defer fakeServer.Close()
address := fakeServer.URL + "/endpoint"
plugin := &httpplugin.HTTP{
URLs: []string{address},
Log: testutil.Logger{},
}
metricName := "metricName"
plugin.SetParserFunc(func() (telegraf.Parser, error) {
p := &json.Parser{MetricName: "metricName"}
err := p.Init()
return p, err
})
var acc testutil.Accumulator
require.NoError(t, plugin.Init())
require.NoError(t, acc.GatherError(plugin.Gather))
require.Len(t, acc.Metrics, 1)
// basic check to see if we got the right field, value and tag
var metric = acc.Metrics[0]
require.Equal(t, metric.Measurement, metricName)
require.Len(t, acc.Metrics[0].Fields, 1)
require.InDelta(t, 1.2, acc.Metrics[0].Fields["a"], testutil.DefaultDelta)
require.Equal(t, acc.Metrics[0].Tags["url"], address)
}
func TestHTTPHeaders(t *testing.T) {
header := "X-Special-Header"
headerValue := "Special-Value"
fakeServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/endpoint" {
if r.Header.Get(header) == headerValue {
if _, err := w.Write([]byte(simpleJSON)); err != nil {
w.WriteHeader(http.StatusInternalServerError)
t.Error(err)
return
}
} else {
w.WriteHeader(http.StatusForbidden)
}
} else {
w.WriteHeader(http.StatusNotFound)
}
}))
defer fakeServer.Close()
address := fakeServer.URL + "/endpoint"
headerSecret := config.NewSecret([]byte(headerValue))
plugin := &httpplugin.HTTP{
URLs: []string{address},
Headers: map[string]*config.Secret{header: &headerSecret},
Log: testutil.Logger{},
}
plugin.SetParserFunc(func() (telegraf.Parser, error) {
p := &json.Parser{MetricName: "metricName"}
err := p.Init()
return p, err
})
var acc testutil.Accumulator
require.NoError(t, plugin.Init())
require.NoError(t, acc.GatherError(plugin.Gather))
}
func TestHTTPContentLengthHeader(t *testing.T) {
fakeServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/endpoint" {
if r.Header.Get("Content-Length") != "" {
if _, err := w.Write([]byte(simpleJSON)); err != nil {
w.WriteHeader(http.StatusInternalServerError)
t.Error(err)
return
}
} else {
w.WriteHeader(http.StatusForbidden)
}
} else {
w.WriteHeader(http.StatusNotFound)
}
}))
defer fakeServer.Close()
address := fakeServer.URL + "/endpoint"
plugin := &httpplugin.HTTP{
URLs: []string{address},
Headers: map[string]*config.Secret{},
Body: "{}",
Log: testutil.Logger{},
}
plugin.SetParserFunc(func() (telegraf.Parser, error) {
p := &json.Parser{MetricName: "metricName"}
err := p.Init()
return p, err
})
var acc testutil.Accumulator
require.NoError(t, plugin.Init())
require.NoError(t, acc.GatherError(plugin.Gather))
}
func TestInvalidStatusCode(t *testing.T) {
fakeServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNotFound)
}))
defer fakeServer.Close()
address := fakeServer.URL + "/endpoint"
plugin := &httpplugin.HTTP{
URLs: []string{address},
Log: testutil.Logger{},
}
plugin.SetParserFunc(func() (telegraf.Parser, error) {
p := &json.Parser{MetricName: "metricName"}
err := p.Init()
return p, err
})
var acc testutil.Accumulator
require.NoError(t, plugin.Init())
require.Error(t, acc.GatherError(plugin.Gather))
}
func TestSuccessStatusCodes(t *testing.T) {
fakeServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusAccepted)
}))
defer fakeServer.Close()
address := fakeServer.URL + "/endpoint"
plugin := &httpplugin.HTTP{
URLs: []string{address},
SuccessStatusCodes: []int{200, 202},
Log: testutil.Logger{},
}
plugin.SetParserFunc(func() (telegraf.Parser, error) {
p := &json.Parser{MetricName: "metricName"}
err := p.Init()
return p, err
})
var acc testutil.Accumulator
require.NoError(t, plugin.Init())
require.NoError(t, acc.GatherError(plugin.Gather))
}
func TestMethod(t *testing.T) {
fakeServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == "POST" {
w.WriteHeader(http.StatusOK)
} else {
w.WriteHeader(http.StatusNotFound)
}
}))
defer fakeServer.Close()
plugin := &httpplugin.HTTP{
URLs: []string{fakeServer.URL},
Method: "POST",
Log: testutil.Logger{},
}
plugin.SetParserFunc(func() (telegraf.Parser, error) {
p := &json.Parser{MetricName: "metricName"}
err := p.Init()
return p, err
})
var acc testutil.Accumulator
require.NoError(t, plugin.Init())
require.NoError(t, acc.GatherError(plugin.Gather))
}
const simpleJSON = `
{
"a": 1.2
}
`
const simpleCSVWithHeader = `
# Simple CSV with header(s)
a,b,c
1.2,3.1415,ok
`
func TestBodyAndContentEncoding(t *testing.T) {
ts := httptest.NewServer(http.NotFoundHandler())
defer ts.Close()
address := "http://" + ts.Listener.Addr().String()
tests := []struct {
name string
plugin *httpplugin.HTTP
queryHandlerFunc func(t *testing.T, w http.ResponseWriter, r *http.Request)
}{
{
name: "no body",
plugin: &httpplugin.HTTP{
Method: "POST",
URLs: []string{address},
Log: testutil.Logger{},
},
queryHandlerFunc: func(t *testing.T, w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(r.Body)
require.NoError(t, err)
require.Equal(t, []byte(""), body)
w.WriteHeader(http.StatusOK)
},
},
{
name: "post body",
plugin: &httpplugin.HTTP{
URLs: []string{address},
Method: "POST",
Body: "test",
Log: testutil.Logger{},
},
queryHandlerFunc: func(t *testing.T, w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(r.Body)
require.NoError(t, err)
require.Equal(t, []byte("test"), body)
w.WriteHeader(http.StatusOK)
},
},
{
name: "get method body is sent",
plugin: &httpplugin.HTTP{
URLs: []string{address},
Method: "GET",
Body: "test",
Log: testutil.Logger{},
},
queryHandlerFunc: func(t *testing.T, w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(r.Body)
require.NoError(t, err)
require.Equal(t, []byte("test"), body)
w.WriteHeader(http.StatusOK)
},
},
{
name: "gzip encoding",
plugin: &httpplugin.HTTP{
URLs: []string{address},
Method: "GET",
Body: "test",
ContentEncoding: "gzip",
Log: testutil.Logger{},
},
queryHandlerFunc: func(t *testing.T, w http.ResponseWriter, r *http.Request) {
require.Equal(t, "gzip", r.Header.Get("Content-Encoding"))
gr, err := gzip.NewReader(r.Body)
require.NoError(t, err)
body, err := io.ReadAll(gr)
require.NoError(t, err)
require.Equal(t, []byte("test"), body)
w.WriteHeader(http.StatusOK)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ts.Config.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
tt.queryHandlerFunc(t, w, r)
})
tt.plugin.SetParserFunc(func() (telegraf.Parser, error) {
parser := &influx.Parser{}
err := parser.Init()
return parser, err
})
var acc testutil.Accumulator
require.NoError(t, tt.plugin.Init())
require.NoError(t, tt.plugin.Gather(&acc))
})
}
}
type testHandlerFunc func(t *testing.T, w http.ResponseWriter, r *http.Request)
func TestOAuthClientCredentialsGrant(t *testing.T) {
ts := httptest.NewServer(http.NotFoundHandler())
defer ts.Close()
var token = "2YotnFZFEjr1zCsicMWpAA"
u, err := url.Parse("http://" + ts.Listener.Addr().String())
require.NoError(t, err)
tests := []struct {
name string
plugin *httpplugin.HTTP
tokenHandler testHandlerFunc
handler testHandlerFunc
}{
{
name: "no credentials",
plugin: &httpplugin.HTTP{
URLs: []string{u.String()},
Log: testutil.Logger{},
},
handler: func(t *testing.T, w http.ResponseWriter, r *http.Request) {
require.Empty(t, r.Header["Authorization"])
w.WriteHeader(http.StatusOK)
},
},
{
name: "success",
plugin: &httpplugin.HTTP{
URLs: []string{u.String() + "/write"},
HTTPClientConfig: common_http.HTTPClientConfig{
OAuth2Config: oauth.OAuth2Config{
ClientID: "howdy",
ClientSecret: "secret",
TokenURL: u.String() + "/token",
Scopes: []string{"urn:opc:idm:__myscopes__"},
},
},
Log: testutil.Logger{},
},
tokenHandler: func(t *testing.T, w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
values := url.Values{}
values.Add("access_token", token)
values.Add("token_type", "bearer")
values.Add("expires_in", "3600")
_, err := w.Write([]byte(values.Encode()))
require.NoError(t, err)
},
handler: func(t *testing.T, w http.ResponseWriter, r *http.Request) {
require.Equal(t, []string{"Bearer " + token}, r.Header["Authorization"])
w.WriteHeader(http.StatusOK)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ts.Config.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/write":
tt.handler(t, w, r)
case "/token":
tt.tokenHandler(t, w, r)
}
})
tt.plugin.SetParserFunc(func() (telegraf.Parser, error) {
p := &value.Parser{
MetricName: "metric",
DataType: "string",
}
err := p.Init()
return p, err
})
err = tt.plugin.Init()
require.NoError(t, err)
var acc testutil.Accumulator
err = tt.plugin.Gather(&acc)
require.NoError(t, err)
})
}
}
func TestHTTPWithCSVFormat(t *testing.T) {
fakeServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/endpoint" {
if _, err := w.Write([]byte(simpleCSVWithHeader)); err != nil {
w.WriteHeader(http.StatusInternalServerError)
t.Error(err)
return
}
} else {
w.WriteHeader(http.StatusNotFound)
}
}))
defer fakeServer.Close()
address := fakeServer.URL + "/endpoint"
plugin := &httpplugin.HTTP{
URLs: []string{address},
Log: testutil.Logger{},
}
plugin.SetParserFunc(func() (telegraf.Parser, error) {
parser := &csv.Parser{
MetricName: "metricName",
SkipRows: 3,
ColumnNames: []string{"a", "b", "c"},
TagColumns: []string{"c"},
}
err := parser.Init()
return parser, err
})
expected := []telegraf.Metric{
testutil.MustMetric("metricName",
map[string]string{
"url": address,
"c": "ok",
},
map[string]interface{}{
"a": 1.2,
"b": 3.1415,
},
time.Unix(0, 0),
),
}
var acc testutil.Accumulator
require.NoError(t, plugin.Init())
require.NoError(t, acc.GatherError(plugin.Gather))
testutil.RequireMetricsEqual(t, expected, acc.GetTelegrafMetrics(), testutil.IgnoreTime())
// Run the parser a second time to test for correct stateful handling
acc.ClearMetrics()
require.NoError(t, acc.GatherError(plugin.Gather))
testutil.RequireMetricsEqual(t, expected, acc.GetTelegrafMetrics(), testutil.IgnoreTime())
}
const (
httpOverUnixScheme = "http+unix"
)
func TestConnectionOverUnixSocket(t *testing.T) {
ts := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/data" {
w.Header().Set("Content-Type", "text/csv")
if _, err := w.Write([]byte(simpleCSVWithHeader)); err != nil {
w.WriteHeader(http.StatusInternalServerError)
t.Error(err)
return
}
} else {
w.WriteHeader(http.StatusNotFound)
}
}))
// The Maximum length of the socket path is 104/108 characters, path created with t.TempDir() is too long for some cases
// (it combines test name with subtest name and some random numbers in the path). Therefore, in this case, it is safer to stick with `os.MkdirTemp()`.
//nolint:usetesting // Ignore "os.TempDir() could be replaced by t.TempDir() in TestConnectionOverUnixSocket" finding.
unixListenAddr := filepath.Join(os.TempDir(), fmt.Sprintf("httptestserver.%d.sock", rand.Intn(1_000_000)))
t.Cleanup(func() { os.Remove(unixListenAddr) })
unixListener, err := net.Listen("unix", unixListenAddr)
require.NoError(t, err)
ts.Listener = unixListener
ts.Start()
defer ts.Close()
// NOTE: Remove ":" from windows filepath and replace all "\" with "/".
// This is *required* so that the unix socket path plays well with unixtransport.
replacer := strings.NewReplacer(":", "", "\\", "/")
sockPath := replacer.Replace(unixListenAddr)
address := fmt.Sprintf("%s://%s:/data", httpOverUnixScheme, sockPath)
plugin := &httpplugin.HTTP{
URLs: []string{address},
Log: testutil.Logger{},
}
plugin.SetParserFunc(func() (telegraf.Parser, error) {
parser := &csv.Parser{
MetricName: "metricName",
SkipRows: 3,
ColumnNames: []string{"a", "b", "c"},
TagColumns: []string{"c"},
}
err := parser.Init()
return parser, err
})
expected := []telegraf.Metric{
testutil.MustMetric("metricName",
map[string]string{
"url": address,
"c": "ok",
},
map[string]interface{}{
"a": 1.2,
"b": 3.1415,
},
time.Unix(22000, 0),
),
}
var acc testutil.Accumulator
require.NoError(t, plugin.Init())
require.NoError(t, acc.GatherError(plugin.Gather))
testutil.RequireMetricsEqual(t, expected, acc.GetTelegrafMetrics(), testutil.IgnoreTime())
// Run the parser a second time to test for correct stateful handling
acc.ClearMetrics()
require.NoError(t, acc.GatherError(plugin.Gather))
testutil.RequireMetricsEqual(t, expected, acc.GetTelegrafMetrics(), testutil.IgnoreTime())
}

88
plugins/inputs/http/sample.conf Normal file
View file

@ -0,0 +1,88 @@
# Read formatted metrics from one or more HTTP endpoints
[[inputs.http]]
## One or more URLs from which to read formatted metrics.
urls = [
"http://localhost/metrics",
"http+unix:///run/user/420/podman/podman.sock:/d/v4.0.0/libpod/pods/json"
]
## HTTP method
# method = "GET"
## Optional HTTP headers
# headers = {"X-Special-Header" = "Special-Value"}
## HTTP entity-body to send with POST/PUT requests.
# body = ""
## HTTP Content-Encoding for write request body, can be set to "gzip" to
## compress body or "identity" to apply no encoding.
# content_encoding = "identity"
## Optional Bearer token settings to use for the API calls.
## Use either the token itself or the token file if you need a token.
# token = "eyJhbGc...Qssw5c"
# token_file = "/path/to/file"
## Optional HTTP Basic Auth Credentials
# username = "username"
# password = "pa$$word"
## OAuth2 Client Credentials. The options 'client_id', 'client_secret', and 'token_url' are required to use OAuth2.
# client_id = "clientid"
# client_secret = "secret"
# token_url = "https://indentityprovider/oauth2/v1/token"
# scopes = ["urn:opc:idm:__myscopes__"]
## HTTP Proxy support
# use_system_proxy = false
# http_proxy_url = ""
## Optional TLS Config
## Set to true/false to enforce TLS being enabled/disabled. If not set,
## enable TLS only if any of the other options are specified.
# tls_enable =
## Trusted root certificates for server
# tls_ca = "/path/to/cafile"
## Used for TLS client certificate authentication
# tls_cert = "/path/to/certfile"
## Used for TLS client certificate authentication
# tls_key = "/path/to/keyfile"
## Password for the key file if it is encrypted
# tls_key_pwd = ""
## Send the specified TLS server name via SNI
# tls_server_name = "kubernetes.example.com"
## Minimal TLS version to accept by the client
# tls_min_version = "TLS12"
## List of ciphers to accept, by default all secure ciphers will be accepted
## See https://pkg.go.dev/crypto/tls#pkg-constants for supported values.
## Use "all", "secure" and "insecure" to add all support ciphers, secure
## suites or insecure suites respectively.
# tls_cipher_suites = ["secure"]
## Renegotiation method, "never", "once" or "freely"
# tls_renegotiation_method = "never"
## Use TLS but skip chain & host verification
# insecure_skip_verify = false
## Optional Cookie authentication
# cookie_auth_url = "https://localhost/authMe"
# cookie_auth_method = "POST"
# cookie_auth_username = "username"
# cookie_auth_password = "pa$$word"
# cookie_auth_headers = { Content-Type = "application/json", X-MY-HEADER = "hello" }
# cookie_auth_body = '{"username": "user", "password": "pa$$word", "authenticate": "me"}'
## cookie_auth_renewal not set or set to "0" will auth once and never renew the cookie
# cookie_auth_renewal = "5m"
## Amount of time allowed to complete the HTTP request
# timeout = "5s"
## List of success status codes
# success_status_codes = [200]
## Data format to consume.
## Each data format has its own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
# data_format = "influx"

65
plugins/inputs/http/sample.conf.in Normal file
View file

@ -0,0 +1,65 @@
# Read formatted metrics from one or more HTTP endpoints
[[inputs.http]]
## One or more URLs from which to read formatted metrics.
urls = [
"http://localhost/metrics",
"http+unix:///run/user/420/podman/podman.sock:/d/v4.0.0/libpod/pods/json"
]
## HTTP method
# method = "GET"
## Optional HTTP headers
# headers = {"X-Special-Header" = "Special-Value"}
## HTTP entity-body to send with POST/PUT requests.
# body = ""
## HTTP Content-Encoding for write request body, can be set to "gzip" to
## compress body or "identity" to apply no encoding.
# content_encoding = "identity"
## Optional Bearer token settings to use for the API calls.
## Use either the token itself or the token file if you need a token.
# token = "eyJhbGc...Qssw5c"
# token_file = "/path/to/file"
## Optional HTTP Basic Auth Credentials
# username = "username"
# password = "pa$$word"
## OAuth2 Client Credentials. The options 'client_id', 'client_secret', and 'token_url' are required to use OAuth2.
# client_id = "clientid"
# client_secret = "secret"
# token_url = "https://indentityprovider/oauth2/v1/token"
# scopes = ["urn:opc:idm:__myscopes__"]
## HTTP Proxy support
# use_system_proxy = false
# http_proxy_url = ""
## Optional TLS Config
{{template "/plugins/common/tls/client.conf"}}
## Optional Cookie authentication
# cookie_auth_url = "https://localhost/authMe"
# cookie_auth_method = "POST"
# cookie_auth_username = "username"
# cookie_auth_password = "pa$$word"
# cookie_auth_headers = { Content-Type = "application/json", X-MY-HEADER = "hello" }
# cookie_auth_body = '{"username": "user", "password": "pa$$word", "authenticate": "me"}'
## cookie_auth_renewal not set or set to "0" will auth once and never renew the cookie
# cookie_auth_renewal = "5m"
## Amount of time allowed to complete the HTTP request
# timeout = "5s"
## List of success status codes
# success_status_codes = [200]
## Data format to consume.
## Each data format has its own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
# data_format = "influx"