Kubernetes

The kubernetes config type scrapes the configurations of your Kubernetes resources as specified with the fields; namespace, selector, fieldSelector and more.

kubernetes-scraper.yaml

apiVersion: configs.flanksource.com/v1
kind: ScrapeConfig
metadata:
  name: kubernetes-scraper
spec:
  kubernetes:
    - clusterName: local-kind-cluster
      exclusions:
        - Secret
        - ReplicaSet
        - APIService
        - endpoints.discovery.k8s.io
        - endpointslices.discovery.k8s.io
        - leases.coordination.k8s.io
        - podmetrics.metrics.k8s.io
        - nodemetrics.metrics.k8s.io
        - customresourcedefinition
        - controllerrevision
        - certificaterequest
        - orders.acme.cert-manager.io
      event:
        exclusions:
          - SuccessfulCreate
          - Created
          - DNSConfigForming
        severityKeywords:
          error:
            - failed
            - error
          warn:
            - backoff
            - nodeoutofmemory

Scraper

Field	Description	Scheme	Required
logLevel	Specify the level of logging.	string
schedule	Specify the interval to scrape in cron format. Defaults to every 60 minutes.	string
retention	Settings for retaining changes, analysis and scraped items	Retention
kubernetes	Specifies the list of Kubernetes configurations to scrape.	[]Kubernetes

Kubernetes

Field	Description	Scheme	Required
allowIncomplete	Show partial results when fetching of API resources fails	bool
clusterName	Specify cluster name	string
event	Specify configuration to handle Kubernetes events.	Event
exclusions	Specify Kubernetes resources to be excluded from scraping	[]string
fieldSelector	Specify Kubernetes resource based on value of resource fields. e.g status.Phase=Running	string
kubeconfig	Specify kubeconfig for access to your Kubernetes Cluster	[]EnvVar
maxInFlight	restrict parallelism to X inflight requests	int64
namespace	Specify namespace for scraping of Kubernetes resources	string
relationships	Create relationships between kubernetes objects.	[]Relationship
scope	Specify scope for scrape. e.g cluster for scraping at Cluster level	string
selector	Specify Kubernetes resource to scrape based on selector. e.g matchLabels	string
since	Set time constraint for scraping resources within the set period	string
useCache	Specify boolean value to toggle fetching results from Kube-apiserver or fetch response from etcd	bool
properties	Custom templatable properties for the scraped config items.	[]ConfigProperty
transform	Field to transform result	Transform
tags	set custom tags on the scraped config items	map[string]string

Transform

Field	Description	Scheme	Required
gotemplate	Go template to transform the scraped config item	string
javascript	Javascript to transform the scraped config item	string
jsonpath	Specify JSONPath	jsonpath
expr	Specify Cel expression	cel-expression
changes	Apply transformation on the scraped changes	[]Changes
exclude	Fields to remove from the config, useful for removing sensitive data and fields that change often without a material impact i.e. Last Scraped Time	[]Exclude
mask	Specify configurations to replace sensitive fields with hash functions or static string.	[]Mask
relationship	Form relationships between config items using selectors	[]RelationshipConfig

info

Both the go template and javascript receive ScrapeResult as the template variable.

Events

Config DB maps Kubernetes Event objects to config changes unlike other objects that are mapped to config items. This configuration allows you to exclude or include the Kubernetes Event objects based on the reason.

In addition, you can also specify keywords used to identify the severity of the Kubernetes Event based on the reason.

Field	Description	Scheme	Required
exclusions	A list of keywords used to exclude event objects based on the reason	[]string
severityKeywords	Specify keywords used to identify the severity of the Kubernetes Event based on the reason	SeverityKeywords

SeverityKeywords

Field	Description	Scheme	Required
warn	A list of keywords used to identify a warning severity from the reason. It could also be a match pattern: example "*" to match all or "!badword" to exclude "badword"	[]string
error	Same as warn but used to map to error severity.	[]string

Kubernetes Relationship

You can create relationships between kubernetes objects on the basis of kind, name and labels. While relationships between node and pod, deployment and pod, namespace and deployment are created automatically, there are cases where we want to link objects on the basis of metadata (like linking resources created by a flux object to it). See Example.

We support static values, expressions and label lookups to find the parent

Field	Description	Scheme	Required
kind	kind of Kubernetes Object	RelationshipLookup	true
name	name of Kubernetes Object	RelationshipLookup	true
namespace	namespace of Kubernetes Object	RelationshipLookup	true

Relationship Lookup

RelationshipLookup offers different ways to specify a lookup value

Field	Description	Scheme	Required
expr	Use an expression to get the value	string
value	Specify a static value	string
label	Get the value from a label	string