OPSaid – Thanos

Docs: Query Frontend

Fri, 21 Oct 2022 12:48:13 +0800

Query Frontend 组件

基于 thanos/v0.28，面向用户层面用于提供查询数据，它是无状态可以进行水平扩展，用于对范围查询与缓存。

当前仅接口/api/v1/query_range会经过该组件处理，其他接口请求会直接传递到下游查询组件。

启动参数

名称	默认值	说明
cache-compression-type	""	对结果集的压缩，支持 ‘snappy’
http-address	0.0.0.0:10902	监听 HTTP 服务地址
http-grace-period	2m	HTTP服务接收到停止需等待时间，通过配置 context.WithTimeout 的实现
http.config	""	TODO
labels.default-time-range	24h	TODO;
labels.max-query-parallelism	14	TODO;
labels.max-retries-per-request	5	TODO;
labels.partial-response	""	如果未指定partial_response参数，则为标签请求启用部分响应
labels.response-cache-config	""	功能同 `labels.response-cache-config-file`
labels.response-cache-config-file	""	TODO;
labels.response-cache-max-freshness	1m	最近允许标签请求的可缓存结果，以防止缓存可能仍在变化中的最近结果
labels.split-interval	24h	按间隔拆分标签请求并并行执行，标签时应大于0。配置了响应缓存配置
log.format	logfmt	日志格式，可取值：logfmt、json
log.level	info	日志级别
log.request.decision	""	已被废弃
query-frontend.compress-responses	""	压缩HTTP响应
query-frontend.downstream-tripper-config	""	功能同 `query-frontend.downstream-tripper-config-file`
query-frontend.downstream-tripper-config-file	TODO；包含downstream tripper的配置，如果是 ‘127.0.0.1’ 则建议 max_idle_conns_per_host 至少配置为100
query-frontend.downstream-url	“http://localhost:9090”	转发至下游兼容 prometheus query api 地址
query-frontend.forward-header	""	转发至下游的http请求头
query-frontend.log-queries-longer-than	0	记录超过配置时间的慢查询，设置0为关闭，设置小于0则开启所有
query-frontend.org-id-header	""	用于标识慢速查询源的请求头名称，将会植入至慢日志’org_id’字段中，如为配置默认植入的值为’anonymous'
query-range.align-range-with-step	""	TODO;
query-range.max-query-length	0	限制查询的时间最大区间范围，设置0为禁用
query-range.max-query-parallelism	14	TODO; 前端将并行计划最大查询范围请求数
query-range.max-retries-per-request	5	TODO; 单个查询范围请求的最大重试次数；除此之外，将返回下游错误
query-range.partial-response	""	如果未指定partial_response参数，则为查询范围请求启用部分响应
query-range.request-downsampled	""	在对范围请求的响应为空或不完整的情况下，对下采样数据进行额外查询
query-range.response-cache-config	""	功能同 `query-range.response-cache-config-file`
query-range.response-cache-config-file	""	响应缓存的配置文件
query-range.response-cache-max-freshness	1m	最近允许查询范围请求的可缓存结果，以防止缓存可能仍在变化中的最近结果
query-range.split-interval	24h	按间隔拆分查询范围请求并并行执行，如果`query-range.response-cache-config`配置了，则必须开启
request.logging-config	""	功能同 `request.logging-config-file`
request.logging-config-file	""	TODO；
tracing.config	""	功能同 `tracing.config-file`
tracing.config-file	""	链路跟踪配置文件
web.disable-cors	""	是否禁用跨域，默认允许所有

功能特性

查询分割

针对一个长查询切割为多个子查询，由启动参数--query-range.split-interval设定，这有以下几个好处：

保护’Query’查询组件，如果是比较大的范围查询有可能导致 OOM 发生；
更好的并行处理；
更好的负载均衡；

失败重试

由启动参数 --query-range.max-retries-per-request 设定，当HTTP请求查询失败时，会进行最大重试次数；

查询缓存

可以对前端查询的结果进行缓存，以便后续复用它们，如果缓存的结果不完整，将计算所需的子查询，并在下游查询器上并行执行它们。查询前端可以选择将查询与其步骤参数对齐，以提高查询结果的可缓存性。目前支持内存缓存（fifo cache）、memcached和redis。

以下几种情况将不会缓存：

不对副本去重，如查询开启参数dedup=false；
开启查询指定存储过滤，Enable Store Filtering；
下游查询组件设置了"Cache-Control=no-store"头；

开启慢日志

由启动参数 --query-frontend.log-queries-longer-than 设置最大允许时间，超过则记录。

Docs: Query

Thu, 20 Oct 2022 21:55:27 +0800

Query 组件

基于 thanos 0.28.1

启动参数

名称	默认值	说明
alert.query-url	-	外部可访问的 Thanos Query URL 地址，以便在告警时可以在 ‘Source’ 字段中展示
enable-feature	query-pushdown	TODO；
endpoint	-	Thanos API servers 的地址，可以重复多个，支持 ‘dns+’ 或 ‘dnssrv+’ 协议自动发现
endpoint-strict	-	同上，明确的地址，不管是否健康探测失败
grpc-address	0.0.0.0:10901	gRPC服务 StoreAPI 监听的地址
grpc-client-server-name	-	grpc认证
grpc-client-tls-ca	-	TODO;
grpc-client-tls-cert	-	TODO;
grpc-client-tls-key	-	TODO;
grpc-client-tls-secure	-	TODO;
grpc-client-tls-skip-verify	-	跳过CA验证，一般在自签证书情况下忽略ca验证
grpc-grace-period	2m	gRPC服务接收到停止需等待时间，通过配置 context.WithTimeout 的实现
grpc-server-max-connection-age	60m	TODO;
grpc-server-tls-cert	""	是否开启tls认证，为空表示禁用
grpc-server-tls-client-ca	""	TLS CA，为空表示不对客户端验证
grpc-server-tls-key	""	是否开启tls认证，为空表示禁用
http-address	0.0.0.0:10902	监听 HTTP 服务地址
http-grace-period	2m	HTTP服务接收到停止需等待时间，通过配置 context.WithTimeout 的实现
http.config	""	TODO
log.format	logfmt	日志格式，可取值：logfmt、json
log.level	info	日志级别
log.request.decision	""	已被废弃
query.active-query-path	""	TODO
query.auto-downsampling	-	如果未指定max_source_resolution参数，则启用自动调整（step/5）以确定存储网关中应使用的数据源
query.default-evaluation-interval	1m	TODO
query.default-step	1s	在 range 查询的默认步长，仅当未在UI上配置时生效。不支持在grafana使用，grafana可支持 __step 变量
query.lookback-delta	5m	当持续存在的指标，后端突然不在返回，需保持原先值的一段时间，比如关闭数据采集需等待这个时间才会看到没有数据
query.max-concurrent	20	TODO; 最大的查询处理请求
query.max-concurrent-select	4	TODO; 最大的 select 查询请求
query.metadata.default-time-range	0s	TODO;
query.partial-response	-	TODO;
query.replica-label	-	用于标识副本的标签，查询的结果集会移除这些标签
query.timeout	2m	最大查询处理时间
request.logging-config	-	TODO；
request.logging-config-file	-	TODO；
selector-label	-	TODO；标签名与值，会自动带到 endpoint 指定的查询地址，可配置多个
store	-	被废弃，由 endpoint 代替
store-strict	-	被废弃，由 `endpoint-strict` 代替
store.response-timeout	0ms	TODO;
store.sd-dns-interval	30s	TODO;
store.sd-files	-	TODO;
store.sd-interval	5m	TODO;
store.unhealthy-timeout	5m	TODO;
tracing.config	-	-
tracing.config-file	-	-
web.disable-cors	-	是否禁止跨域，默认是允许
web.external-prefix	-	静态资源在html中的前缀
web.prefix-header	-	功能同上，根据http请求头去设置前缀，不能与上面同时使用
web.route-prefix	-	接口前缀

Docs: Receiver

Sat, 08 Oct 2022 15:31:20 +0800

参数解析

启动参数

名称	默认值	说明
grpc-address	0.0.0.0:10901	gRPC服务 StoreAPI 监听的地址
grpc-grace-period	2m	gRPC服务接收到停止需等待时间，通过配置 context.WithTimeout 的实现
grpc-server-max-connection-age	60m	TODO;
grpc-server-tls-cert	""	是否开启tls认证，为空表示禁用
grpc-server-tls-client-ca	""	TLS CA，为空表示不对客户端验证
grpc-server-tls-key	""	是否开启tls认证，为空表示禁用
hash-func	""	TODO; 如："", “SHA256”
http-address	0.0.0.0:10902	监听 HTTP 服务地址
http-grace-period	2m	HTTP服务接收到停止需等待时间，通过配置 context.WithTimeout 的实现
http.config	""	TODO; http的配置，用于开启TLS、认证等
label	""	格式：=""，植入标签，用于标识
log.format	logfmt	日志格式，可取值：logfmt、json
log.level	info	日志级别
objstore.config	""	功能同 ‘objstore.config-file’ 对象存储地址
objstore.config-file	""	对象存储地址
receive.default-tenant-id	default-tenant	如果未设置租户ID，则以此为默认值
receive.hashrings	""	功能同 ‘receive.hashrings-file’，优先级高
receive.hashrings-algorithm	hashmod	用于 hashrings 算法，可取：hashmod, ketama
receive.hashrings-file	""	TODO;
receive.hashrings-file-refresh-interval	5m	刷新 hashrings-file 指向文件的时间
receive.local-endpoint	""	在hashring文件列表配置的所有地址，指明哪个为该节点，如果为空且配置了hashring则开启"RoutingOnly"模式
receive.relabel-config	""	功能同 ‘receive.relabel-config-file’
receive.relabel-config-file	""	TODO;
receive.replica-header	THANOS-REPLICA	TODO; 用于表示副本数的HTTP请求头
receive.replication-factor	1	TODO; int((h.options.ReplicationFactor / 2) + 1)
receive.tenant-certificate-field	""	使用 tls 证书中的属性表示租户，必须是 organization, organizationalUnit or commonName 之一，优先级高于 receive.tenant-header
receive.tenant-header	THANOS-TENANT	HTTP请求头，值代表写入的租户
receive.tenant-label-name	tenant_id	植入到指标里租户的标签名
remote-write.address	0.0.0.0:19291	用于 remote write 的http服务地址
remote-write.client-server-name	""	TODO;
remote-write.client-tls-ca	""	用于验证客户端
remote-write.client-tls-cert	""	同上
remote-write.client-tls-key	""	同上
remote-write.server-tls-cert	""	用于 remote write 服务是否开启tls，为空表示关闭
remote-write.server-tls-key	""	同上
remote-write.server-tls-client-ca	""	用于验证客户端，未配置则不开启
request.logging-config	""	功能同 ‘request.logging-config-file’
request.logging-config-file	""	TODO;
tracing.config	""	功能同 ’tracing.config-file’
tracing.config-file	""	TODO;
tsdb.allow-overlapping-blocks	-	TODO;
tsdb.max-exemplars	0	TODO;
tsdb.no-lockfile	""	TODO;
tsdb.path	./data	TODO;
tsdb.retention	15d	性能数据保留的时间，设置0表示不过期
tsdb.wal-compression	""	是否压缩 wal

隐藏参数

名称	默认值	说明
receive-forward-timeout	5s	转发的超时时间
tsdb.min-block-duration	2h	TODO;
tsdb.max-block-duration	2h	TODO;
shipper.ignore-unequal-block-size	false	TODO;
shipper.allow-out-of-order-uploads	false	TODO;

两种模式

Routing Receiver

在启动参数不配置以下时，为该模式，仅做流量转发

--receive.local-endpoint=RECEIVE.LOCAL-ENDPOINT

该模式需明确指定--receive.hashrings-file参数。

Ingesting Receiver

在启动参数不配置以下时，为该模式，仅做数据存储

--receive.hashrings-file=<path>

该模式需明确指定--receive.local-endpoint参数。

RouterIngestor

既是Routing也是Ingesting

// determineMode returns the ReceiverMode that this receiver is configured to run in.
// This is used to configure this Receiver's forwarding and ingesting behavior at runtime.
func (rc *receiveConfig) determineMode() receive.ReceiverMode {
 // Has the user provided some kind of hashring configuration?
 hashringSpecified := rc.hashringsFileContent != "" || rc.hashringsFilePath != ""
 // Has the user specified the --receive.local-endpoint flag?
 localEndpointSpecified := rc.endpoint != ""

 switch {
 case hashringSpecified && localEndpointSpecified:
 return receive.RouterIngestor
 case hashringSpecified && !localEndpointSpecified:
 // Be careful - if the hashring contains an address that routes to itself and does not specify a local
 // endpoint - you've just created an infinite loop / fork bomb :)
 return receive.RouterOnly
 default:
 // hashring configuration has not been provided so we ingest all metrics locally.
 return receive.IngestorOnly
 }
}

Docs: Config

Thu, 29 Sep 2022 15:47:36 +0800

公共资源配置

alertmanagers

配置实例

alertmanagers:
- http_config:
 basic_auth:
 username: ""
 password: ""
 password_file: ""
 bearer_token: ""
 bearer_token_file: ""
 proxy_url: ""
 tls_config:
 ca_file: ""
 cert_file: ""
 key_file: ""
 server_name: ""
 insecure_skip_verify: false
 static_configs: []
 file_sd_configs:
 - files: []
 refresh_interval: 0s
 scheme: http
 path_prefix: ""
 timeout: 10s
 api_version: v2

数据结构

type AlertingConfig struct {
 Alertmanagers []AlertmanagerConfig `yaml:"alertmanagers"`
}

// AlertmanagerConfig represents a client to a cluster of Alertmanager endpoints.
type AlertmanagerConfig struct {
 HTTPClientConfig httpconfig.ClientConfig `yaml:"http_config"`
 EndpointsConfig httpconfig.EndpointsConfig `yaml:",inline"`
 Timeout model.Duration `yaml:"timeout"`
 APIVersion APIVersion `yaml:"api_version"`
}

httpconfig.ClientConfig

// ClientConfig configures an HTTP client.
type ClientConfig struct {
 // The HTTP basic authentication credentials for the targets.
 BasicAuth BasicAuth `yaml:"basic_auth"`
 // The bearer token for the targets.
 BearerToken string `yaml:"bearer_token"`
 // The bearer token file for the targets.
 BearerTokenFile string `yaml:"bearer_token_file"`
 // HTTP proxy server to use to connect to the targets.
 ProxyURL string `yaml:"proxy_url"`
 // TLSConfig to use to connect to the targets.
 TLSConfig TLSConfig `yaml:"tls_config"`
 // TransportConfig for Client transport properties
 TransportConfig TransportConfig `yaml:"transport_config"`
 // ClientMetrics contains metrics that will be used to instrument
 // the client that will be created with this config.
 ClientMetrics *extpromhttp.ClientMetrics `yaml:"-"`
}

// BasicAuth configures basic authentication for HTTP clients.
type BasicAuth struct {
 Username string `yaml:"username"`
 Password string `yaml:"password"`
 PasswordFile string `yaml:"password_file"`
}

// TLSConfig configures TLS connections.
type TLSConfig struct {
 // The CA cert to use for the targets.
 CAFile string `yaml:"ca_file"`
 // The client cert file for the targets.
 CertFile string `yaml:"cert_file"`
 // The client key file for the targets.
 KeyFile string `yaml:"key_file"`
 // Used to verify the hostname for the targets. See https://tools.ietf.org/html/rfc4366#section-3.1
 ServerName string `yaml:"server_name"`
 // Disable target certificate validation.
 InsecureSkipVerify bool `yaml:"insecure_skip_verify"`
}

// Transport configures client's transport properties.
type TransportConfig struct {
 MaxIdleConns int `yaml:"max_idle_conns"`
 MaxIdleConnsPerHost int `yaml:"max_idle_conns_per_host"`
 IdleConnTimeout int64 `yaml:"idle_conn_timeout"`
 ResponseHeaderTimeout int64 `yaml:"response_header_timeout"`
 ExpectContinueTimeout int64 `yaml:"expect_continue_timeout"`
 MaxConnsPerHost int `yaml:"max_conns_per_host"`
 DisableCompression bool `yaml:"disable_compression"`
 TLSHandshakeTimeout int64 `yaml:"tls_handshake_timeout"`
}

httpconfig.EndpointsConfig

// EndpointsConfig configures a cluster of HTTP endpoints from static addresses and
// file service discovery.
type EndpointsConfig struct {
 // List of addresses with DNS prefixes.
 StaticAddresses []string `yaml:"static_configs"`
 // List of file configurations (our FileSD supports different DNS lookups).
 FileSDConfigs []FileSDConfig `yaml:"file_sd_configs"`

 // The URL scheme to use when talking to targets.
 Scheme string `yaml:"scheme"`

 // Path prefix to add in front of the endpoint path.
 PathPrefix string `yaml:"path_prefix"`
}

// FileSDConfig represents a file service discovery configuration.
type FileSDConfig struct {
 Files []string `yaml:"files"`
 RefreshInterval model.Duration `yaml:"refresh_interval"`
}

objstore

配置示例

type: S3
config:
 bucket: ""
 endpoint: ""
 region: ""
 aws_sdk_auth: false
 access_key: ""
 insecure: false
 signature_version2: false
 secret_key: ""
 put_user_metadata: {}
 http_config:
 idle_conn_timeout: 1m30s
 response_header_timeout: 2m
 insecure_skip_verify: false
 tls_handshake_timeout: 10s
 expect_continue_timeout: 1s
 max_idle_conns: 100
 max_idle_conns_per_host: 100
 max_conns_per_host: 0
 tls_config:
 ca_file: ""
 cert_file: ""
 key_file: ""
 server_name: ""
 insecure_skip_verify: false
 disable_compression: false
 trace:
 enable: false
 list_objects_version: ""
 bucket_lookup_type: auto
 part_size: 67108864
 sse_config:
 type: ""
 kms_key_id: ""
 kms_encryption_context: {}
 encryption_key: ""
 sts_endpoint: ""
prefix: ""

数据结构

type ObjProvider string

const (
 FILESYSTEM ObjProvider = "FILESYSTEM"
 GCS ObjProvider = "GCS"
 S3 ObjProvider = "S3"
 AZURE ObjProvider = "AZURE"
 SWIFT ObjProvider = "SWIFT"
 COS ObjProvider = "COS"
 ALIYUNOSS ObjProvider = "ALIYUNOSS"
 BOS ObjProvider = "BOS"
)

type BucketConfig struct {
 Type ObjProvider `yaml:"type"`
 Config interface{} `yaml:"config"`
 Prefix string `yaml:"prefix" default:""`
}

ObjProvider 为 FILESYSTEM 的 Config

// Config stores the configuration for storing and accessing blobs in filesystem.
type Config struct {
 Directory string `yaml:"directory"`
}

ObjProvider 为 S3 的 Config

// Config stores the configuration for s3 bucket.
type Config struct {
 Bucket string `yaml:"bucket"`
 Endpoint string `yaml:"endpoint"`
 Region string `yaml:"region"`
 AWSSDKAuth bool `yaml:"aws_sdk_auth"`
 AccessKey string `yaml:"access_key"`
 Insecure bool `yaml:"insecure"`
 SignatureV2 bool `yaml:"signature_version2"`
 SecretKey string `yaml:"secret_key"`
 PutUserMetadata map[string]string `yaml:"put_user_metadata"`
 HTTPConfig HTTPConfig `yaml:"http_config"`
 TraceConfig TraceConfig `yaml:"trace"`
 ListObjectsVersion string `yaml:"list_objects_version"`
 BucketLookupType BucketLookupType `yaml:"bucket_lookup_type"`
 // PartSize used for multipart upload. Only used if uploaded object size is known and larger than configured PartSize.
 // NOTE we need to make sure this number does not produce more parts than 10 000.
 PartSize uint64 `yaml:"part_size"`
 SSEConfig SSEConfig `yaml:"sse_config"`
 STSEndpoint string `yaml:"sts_endpoint"`
}

ObjProvider 为 COS 的 Config

// Config encapsulates the necessary config values to instantiate an cos client.
type Config struct {
 Bucket string `yaml:"bucket"`
 Region string `yaml:"region"`
 AppId string `yaml:"app_id"`
 Endpoint string `yaml:"endpoint"`
 SecretKey string `yaml:"secret_key"`
 SecretId string `yaml:"secret_id"`
 HTTPConfig HTTPConfig `yaml:"http_config"`
}

ObjProvider 为 ALIYUNOSS 的 Config

// Config stores the configuration for oss bucket.
type Config struct {
 Endpoint string `yaml:"endpoint"`
 Bucket string `yaml:"bucket"`
 AccessKeyID string `yaml:"access_key_id"`
 AccessKeySecret string `yaml:"access_key_secret"`
}

query.config

配置示例

- http_config:
 basic_auth:
 username: ""
 password: ""
 password_file: ""
 bearer_token: ""
 bearer_token_file: ""
 proxy_url: ""
 tls_config:
 ca_file: ""
 cert_file: ""
 key_file: ""
 server_name: ""
 insecure_skip_verify: false
 static_configs: []
 file_sd_configs:
 - files: []
 refresh_interval: 0s
 scheme: http
 path_prefix: ""

数据结构

var queryCfg []httpconfig.Config
var err error
if len(conf.queryConfigYAML) > 0 {
 queryCfg, err = httpconfig.LoadConfigs(conf.queryConfigYAML)
 if err != nil {
 return err
 }
}

httpconfig.Config

// Config is a structure that allows pointing to various HTTP endpoint, e.g ruler connecting to queriers.
type Config struct {
 HTTPClientConfig ClientConfig `yaml:"http_config"`
 EndpointsConfig EndpointsConfig `yaml:",inline"`
}

其中 ClientConfig 如上。

remote-write.config

配置示例

remote_write:
- url: http://10.59.107.169:19291/api/v1/receive
 name: thanos-receiver

数据结构

var rwCfg struct {
 RemoteWriteConfigs []*config.RemoteWriteConfig `yaml:"remote_write,omitempty"`
}

config.RemoteWriteConfig

// RemoteWriteConfig is the configuration for writing to remote storage.
type RemoteWriteConfig struct {
 URL *config.URL `yaml:"url"`
 RemoteTimeout model.Duration `yaml:"remote_timeout,omitempty"`
 Headers map[string]string `yaml:"headers,omitempty"`
 WriteRelabelConfigs []*relabel.Config `yaml:"write_relabel_configs,omitempty"`
 Name string `yaml:"name,omitempty"`
 SendExemplars bool `yaml:"send_exemplars,omitempty"`

 // We cannot do proper Go type embedding below as the parser will then parse
 // values arbitrarily into the overflow maps of further-down types.
 HTTPClientConfig config.HTTPClientConfig `yaml:",inline"`
 QueueConfig QueueConfig `yaml:"queue_config,omitempty"`
 MetadataConfig MetadataConfig `yaml:"metadata_config,omitempty"`
 SigV4Config *sigv4.SigV4Config `yaml:"sigv4,omitempty"`
}

request.logging-config

配置示例

http:
 config:
 - path: /api/v1/query
 port: 10904
 options:
 level: DEBUG
 decision:
 log_start: true
 log_end: true

grpc:
 config:
 - service: thanos.Store
 method: Info
 options:
 level: ERROR
 decision:
 log_start: false
 log_end: true

数据结构

type RequestConfig struct {
 HTTP HTTPProtocolConfigs `yaml:"http"`
 GRPC GRPCProtocolConfigs `yaml:"grpc"`
 Options OptionsConfig `yaml:"options"`
}

type HTTPProtocolConfigs struct {
 Options OptionsConfig `yaml:"options"`
 Config []HTTPProtocolConfig `yaml:"config"`
}

type GRPCProtocolConfigs struct {
 Options OptionsConfig `yaml:"options"`
 Config []GRPCProtocolConfig `yaml:"config"`
}

type OptionsConfig struct {
 Level string `yaml:"level"`
 Decision DecisionConfig `yaml:"decision"`
}

type DecisionConfig struct {
 LogStart bool `yaml:"log_start"`
 LogEnd bool `yaml:"log_end"`
}

type HTTPProtocolConfig struct {
 Path string `yaml:"path"`
 Port uint64 `yaml:"port"`
}

type GRPCProtocolConfig struct {
 Service string `yaml:"service"`
 Method string `yaml:"method"`
}

tracing.config

配置示例

type: JAEGER
config:
 service_name: ""
 disabled: false
 rpc_metrics: false
 tags: ""
 sampler_type: ""
 sampler_param: 0
 sampler_manager_host_port: ""
 sampler_max_operations: 0
 sampler_refresh_interval: 0s
 reporter_max_queue_size: 0
 reporter_flush_interval: 0s
 reporter_log_spans: false
 endpoint: ""
 user: ""
 password: ""
 agent_host: ""
 agent_port: 0
 traceid_128bit: false

数据结构

type TracingProvider string

const (
 Stackdriver TracingProvider = "STACKDRIVER"
 GoogleCloud TracingProvider = "GOOGLE_CLOUD"
 Jaeger TracingProvider = "JAEGER"
 ElasticAPM TracingProvider = "ELASTIC_APM"
 Lightstep TracingProvider = "LIGHTSTEP"
)

type TracingConfig struct {
 Type TracingProvider `yaml:"type"`
 Config interface{} `yaml:"config"`
}

TracingProvider 为 Jaeger 的 Config

// Config - YAML configuration. For details see to https://github.com/jaegertracing/jaeger-client-go#environment-variables.
type Config struct {
 ServiceName string `yaml:"service_name"`
 Disabled bool `yaml:"disabled"`
 RPCMetrics bool `yaml:"rpc_metrics"`
 Tags string `yaml:"tags"`
 SamplerType string `yaml:"sampler_type"`
 SamplerParam float64 `yaml:"sampler_param"`
 SamplerManagerHostPort string `yaml:"sampler_manager_host_port"`
 SamplerMaxOperations int `yaml:"sampler_max_operations"`
 SamplerRefreshInterval time.Duration `yaml:"sampler_refresh_interval"`
 ReporterMaxQueueSize int `yaml:"reporter_max_queue_size"`
 ReporterFlushInterval time.Duration `yaml:"reporter_flush_interval"`
 ReporterLogSpans bool `yaml:"reporter_log_spans"`
 Endpoint string `yaml:"endpoint"`
 User string `yaml:"user"`
 Password string `yaml:"password"`
 AgentHost string `yaml:"agent_host"`
 AgentPort int `yaml:"agent_port"`
 Gen128Bit bool `yaml:"traceid_128bit"`
}

cache.conf

配置示例

memory

type: IN-MEMORY
config:
 max_size: ""
 max_size_items: 0
 validity: 0s

memcached

type: MEMCACHED
config:
 addresses: []
 timeout: 0s
 max_idle_connections: 0
 max_async_concurrency: 0
 max_async_buffer_size: 0
 max_get_multi_concurrency: 0
 max_item_size: 0
 max_get_multi_batch_size: 0
 dns_provider_update_interval: 0s
 auto_discovery: false
 expiration: 0s

redis

type: REDIS
config:
 addr: ""
 username: ""
 password: ""
 db: 0
 dial_timeout: 5s
 read_timeout: 3s
 write_timeout: 3s
 pool_size: 100
 min_idle_conns: 10
 idle_timeout: 5m0s
 max_conn_age: 0s
 max_get_multi_concurrency: 100
 get_multi_batch_size: 100
 max_set_multi_concurrency: 100
 set_multi_batch_size: 100
 tls_enabled: false
 tls_config:
 ca_file: ""
 cert_file: ""
 key_file: ""
 server_name: ""
 insecure_skip_verify: false
 expiration: 24h0m0s

数据结构

type ResponseCacheProvider string

const (
 INMEMORY ResponseCacheProvider = "IN-MEMORY"
 MEMCACHED ResponseCacheProvider = "MEMCACHED"
 REDIS ResponseCacheProvider = "REDIS"
)

// CacheProviderConfig is the initial CacheProviderConfig struct holder before parsing it into a specific cache provider.
// Based on the config type the config is then parsed into a specific cache provider.
type CacheProviderConfig struct {
 Type ResponseCacheProvider `yaml:"type"`
 Config interface{} `yaml:"config"`
}

ResponseCacheProvider 为 IN-MEMORY 的 Config

// InMemoryResponseCacheConfig holds the configs for the in-memory cache provider.
type InMemoryResponseCacheConfig struct {
 // MaxSize represents overall maximum number of bytes cache can contain.
 MaxSize string `yaml:"max_size"`
 // MaxSizeItems represents the maximum number of entries in the cache.
 MaxSizeItems int `yaml:"max_size_items"`
 // Validity represents the expiry duration for the cache.
 Validity time.Duration `yaml:"validity"`
}

ResponseCacheProvider 为 MEMCACHED 的 Config

// MemcachedResponseCacheConfig holds the configs for the memcache cache provider.
type MemcachedResponseCacheConfig struct {
 Memcached cacheutil.MemcachedClientConfig `yaml:",inline"`
 // Expiration sets a global expiration limit for all cached items.
 Expiration time.Duration `yaml:"expiration"`
}

ResponseCacheProvider 为 REDIS 的 Config

// RedisResponseCacheConfig holds the configs for the redis cache provider.
type RedisResponseCacheConfig struct {
 Redis cacheutil.RedisClientConfig `yaml:",inline"`
 // Expiration sets a global expiration limit for all cached items.
 Expiration time.Duration `yaml:"expiration"`
}

// RedisClientConfig is the config accepted by RedisClient.
type RedisClientConfig struct {
 // Addr specifies the addresses of redis server.
 Addr string `yaml:"addr"`

 // Use the specified Username to authenticate the current connection
 // with one of the connections defined in the ACL list when connecting
 // to a Redis 6.0 instance, or greater, that is using the Redis ACL system.
 Username string `yaml:"username"`
 // Optional password. Must match the password specified in the
 // requirepass server configuration option (if connecting to a Redis 5.0 instance, or lower),
 // or the User Password when connecting to a Redis 6.0 instance, or greater,
 // that is using the Redis ACL system.
 Password string `yaml:"password"`

 // DB Database to be selected after connecting to the server.
 DB int `yaml:"db"`

 // DialTimeout specifies the client dial timeout.
 DialTimeout time.Duration `yaml:"dial_timeout"`

 // ReadTimeout specifies the client read timeout.
 ReadTimeout time.Duration `yaml:"read_timeout"`

 // WriteTimeout specifies the client write timeout.
 WriteTimeout time.Duration `yaml:"write_timeout"`

 // Maximum number of socket connections.
 PoolSize int `yaml:"pool_size"`

 // MinIdleConns specifies the minimum number of idle connections which is useful when establishing
 // new connection is slow.
 MinIdleConns int `yaml:"min_idle_conns"`

 // Amount of time after which client closes idle connections.
 // Should be less than server's timeout.
 // -1 disables idle timeout check.
 IdleTimeout time.Duration `yaml:"idle_timeout"`

 // Connection age at which client retires (closes) the connection.
 // Default 0 is to not close aged connections.
 MaxConnAge time.Duration `yaml:"max_conn_age"`

 // MaxGetMultiConcurrency specifies the maximum number of concurrent GetMulti() operations.
 // If set to 0, concurrency is unlimited.
 MaxGetMultiConcurrency int `yaml:"max_get_multi_concurrency"`

 // GetMultiBatchSize specifies the maximum size per batch for mget.
 GetMultiBatchSize int `yaml:"get_multi_batch_size"`

 // MaxSetMultiConcurrency specifies the maximum number of concurrent SetMulti() operations.
 // If set to 0, concurrency is unlimited.
 MaxSetMultiConcurrency int `yaml:"max_set_multi_concurrency"`

 // SetMultiBatchSize specifies the maximum size per batch for pipeline set.
 SetMultiBatchSize int `yaml:"set_multi_batch_size"`
}

Docs: Rule

Tue, 27 Sep 2022 10:14:03 +0800

Rule 组件

基于版本 thanos-0.27.0 说明

参数解析

命令行参数

名称	默认值	说明
alert.label-drop	-	发送至 alertmanager 组件需要丢弃的标签，一般用于丢弃多副本的标签，如：replica
alert.query-url	-	外部可访问的 Thanos Query URL 地址，以便在告警时可以在 ‘Source’ 字段中展示
alert.relabel-config	-	功能同 ‘alert.relabel-config-file’，但这里直接添加YAML内容
alert.relabel-config-file	-	同上
alertmanagers.config	-	功能同 ‘alertmanagers.config-file’
alertmanagers.config-file	-	连接的 alertmanager 组件配置
alertmanagers.sd-dns-interval	30s	alertmanager 组件地址通过 dns 协议发现的刷新时间间隔
alertmanagers.send-timeout	10s	发送 alertmanager 组件超时时间
alertmanagers.url	-	alertmanager 组件地址，可支持协议：http、dns+、dnssrv+
data-dir	data/	TODO
eval-interval	1m	默认告警规则评估时间间隔
grpc-address	0.0.0.0:10901	gRPC服务 StoreAPI 监听的地址
grpc-grace-period	2m	gRPC服务接收到停止需等待时间，通过配置 context.WithTimeout 的实现
grpc-server-tls-cert	""	是否开启tls认证，为空表示禁用
grpc-server-tls-client-ca	""	TLS CA，为空表示不对客户端验证
grpc-server-tls-key	""	是否开启tls认证，为空表示禁用
hash-func	""	TODO; 如："", “SHA256”
http-address	0.0.0.0:10902	监听 HTTP 服务地址
http-grace-period	2m	HTTP服务接收到停止需等待时间，通过配置 context.WithTimeout 的实现
http.config	""	TODO; http的配置，用于开启TLS、认证等
label	""	格式：=""，植入标签，用于标识
log.format	logfmt	日志格式，可取值：logfmt、json
log.level	info	日志级别
log.request.decision	-	已被废弃
objstore.config	""	功能同 ‘objstore.config-file’ 对象存储地址
objstore.config-file	""	对象存储地址
query	""	用于查询指标的地址，可支持协议：http、dns+、dnssrv+
query.config	""	功能同 ‘query’、‘query.config-file’，优先级最高
query.config-file	""	功能同 ‘query’、‘query.config’
query.http-method	POST	查询请求的HTTP方法，支持：GET、POST
query.sd-dns-interval	30s	query 组件地址通过 dns 协议发现的刷新时间间隔
query.sd-files	""	基于文件形式的自动发现
query.sd-interval	5m	刷新基于文件形式的自动发现的时间间隔
remote-write.config	""	写入数据至远程，rule 组件本地不会存储任何数据
remote-write.config-file	""	功能同 ‘remote-write.config’
request.logging-config	""	请求日志配置
request.logging-config-file	""	功能同 ‘request.logging-config’
resend-delay	1m	TODO;
rule-file	rules/	告警规则目录
shipper.upload-compacted	""	TODO; 一般用于迁移目的，首次开启后需关闭
tracing.config	""	opentracing 组件配置
tracing.config-file	""	功能同 ’tracing.config’
tsdb.block-duration	2h	TSDB 块时间
tsdb.no-lockfile	""	不创建tsdb lockfile 文件
tsdb.retention	48h	tsdb 最大持久时间
tsdb.wal-compression	""	是否对 tsdb 的 WAL 文件压缩
web.disable-cors	""	是否禁止 CORS，默认允许
web.external-prefix	""	HTML页面上添加的URL前缀，默认为 ‘/’
web.prefix-header	""	TODO
web.route-prefix	""	API接口上添加的URL前缀，默认为’/’，一般同时更改 ‘web.external-prefix’