Kubernetes Log Format
This guide describes how to send logs from your Kubernetes cluster to your Humio cluster. If you're looking to install Humio on Kubernetes, see our Installation Guide.
When it comes to managing micro-services in a Kubernetes cluster, Humio is a great way to get insights into your applications. While other log shippers are supported, we focus on using Fluent Bit for forwarding log messages to Humio.
Helm
Take advantage of Humio with your Kubernetes setup.
We'll start with Helm, the Kubernetes package manager. Directions for installing Helm for your particular OS are on the Helm GitHub page.
Once you've installed Helm, update the main Helm chart repository. This main repository contains subcharts for Humio.
helm repo add humio https://humio.github.io/humio-helm-charts
helm repo update
Next, create a file named humio-agent.yaml
with the
following content:
humio-fluentbit:
enabled: true
humioHostname: $YOUR_HUMIO_URL
es:
tls: true
If using OpenShift, it is necessary to create the following
humio-agent.yaml
with the extra fields which will
create a SecurityContextConstraints
resource as well
as run fluent-bit in a privileged securityContext:
humio-fluentbit:
enabled: true
humioHostname: $YOUR_HUMIO_URL
es:
tls: true
securityContext:
privileged: true
scc:
enabled: true
If using CRI rather than docker, it is necessary to create the following
humio-agent.yaml
with an
inputConfig
and parserConfig
so
that logs are correctly parsed by fluentbit.
humio-fluentbit:
enabled: true
humioHostname: $YOUR_HUMIO_URL
es:
tls: true
parserConfig: |-
[PARSER]
Name cri
Format regex
Regex ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*) (?<log>.*)$
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L%z
inputConfig: |-
[INPUT]
Name tail
Path /var/log/containers/*.log
Parser cri
Tag kube.*
Refresh_Interval 5
Mem_Buf_Limit 256MB
Skip_Long_Lines On
Replace $YOUR_HUMIO_URL with the
hostname of your Humio installation, not including the port number. E.g.
$YOUR_HUMIO_URL
may become
cloud.humio.com
. Don't forget to enable the
ELASTIC_PORT
property. Take your ingest token from your
Humio Repository page.

Figure 221. Token
-- and replace $INGEST_TOKEN
with the ingest token
when running the install command below. We recommend running in a
separate namespace, in our example the logging namespace.
# Helm v3+
helm install humio humio/humio-helm-charts \
--namespace logging \
--set humio-fluentbit.token=$INGEST_TOKEN \
--values humio-agent.yml
# Helm v2
helm install humio/humio-helm-charts --name humio \
--namespace logging \
--set humio-fluentbit.token=$INGEST_TOKEN \
--values humio-agent.yaml
Once this is in place, the Kubernetes logs should be flowing into the Humio cluster.
Parsing Logs
Humio will look for the
pod
label humio-parser
and use that parser on
the log line. That way you can control how Humio parses the data by
configuring your pods. If the humio-parser
label is
not set, no parser will be applied to the log lines.
Additional Filters
In some cases you might want to make some changes to the Fluent Bit configuration. The easiest way to do that is by updating the values.yaml with the filter and updating the Helm chart.
First, update the values.yaml by adding a
customFluentBitConfig
section containing custom
Fluent Bit configurations:
humio-fluentbit:
enabled: true
humioHostname: $YOUR_HUMIO_URL
es:
tls: true
customFluentBitConfig:
custom-filter-kubernetes.conf: |-
[FILTER]
Name modify
Match *
Rename log rawstring
Then upgrade the helm chart
helm upgrade humio humio/humio-helm-charts \
--values humio-agent.yaml
Uninstalling
helm delete --purge humio
And optionally clean up the namespace
kubectl delete namespace logging --cascade=true
For more information see Advanced log routing with Fluent Bit 1.4.