When it comes to managing micro-services in a Kubernetes cluster, LogScale is a great way to get insights into your applications.

The LogScale Collector can be deployed in a Kubernetes to forwarding log messages from the applications deployed in the cluster.

In case of e.g. an application crashing on a virtual machine, the logs from the application are still available until deleted. In Kubernetes, when pods crash, are deleted or scheduled on a new node, the logs from the application containers are lost. For this reason if you want insight into e.g. why a crash occurred, you need the logs forwarded to a centralized log management solution like e.g. LogScale.

Several different deployments are possible, but the model below describes node-level logging using the DaemonSet model. (This is delivered as an out-of-the-box solution consisting of a LogScale Collector Helm chart and container image) DaemonSet approach in which a node-level LogScale Collector runs on every node, and handles logging for all application containers in pods on the node.

Figure 42. Node Level Logging

In this scenario the LogScale Collector is deployed as a DaemonSet on a Kubernetes node to ingest logs from applications running in pods on that node.

The LogScale collector is deployed as a Kubernetes DaemonSet, which is a Kubernetes feature that lets you run a Kubernetes pod on all cluster nodes that meet certain criteria. Every time a new node is added to a cluster, the pod is added to it, and when a node is removed from the cluster, the pod is removed.

Node-level logging creates one LogScale Collector per node and does not require any changes to the applications running on the node.

Containers write to stdout and stderr, but with no agreed format. A node-level LogScale Collector collects these logs and forwards them in realtime to LogScale for live analysis and storage/future analysis.

This is accomplished by running the LogScale Collector in a container that has access to a directory with log files from all of the application containers in all pods on that node.

CrowdStrike provides a LogScale Collector Helm chart for deploying the Logscale Collector in Kubernetes as a DaemonSet, collecting logs from pods.

A Helm chart is a package that contains all the necessary resources to deploy an application to a Kubernetes cluster. This includes YAML configuration files for deployments, services, secrets, and configurations maps that define the desired state of your application.

Helm uses a packaging format called charts. A chart is a collection of files that describe a related set of Kubernetes resources.

Directions for installing Helm for your particular OS can be found here: https://github.com/helm/helm page.

sources:
  containers:
    type: file
      include: /var/log/containers/*.log
      exclude: /var/log/containers/{{ .Release.Name }}-logscale-collector-*.log
      transforms:
        - type: kubernetes
      sink: humio
      {{- if .Values.collectJournal }}
      journal:
        type: journald
        sink: humio
      {{- end }}

kube-apiserver-docker-desktop_kube-system_kube-apiserver-479ebd6ee0267ba8b215e90d389517be2f4896cb42340e5ba05f357e213ffed2.log

^([^_]+)_([^_]+)_(.+)-([A-Fa-f0-9]+)\.log$

Pod Name: kube-apiserver-docker-desktop
        Namespace: kube-system
        Container name: kube-apiserver
        Container ID: 479ebd6ee0267ba8b215e90d389517be2f4896cb42340e5ba05f357e213ffed2

You can add additional fields to the logs collected using the Helm chart installation. The fields are added using the static_fields transform, and support environment variable expansion. Additionally, it's now possible to augment the environment using the Kubernetes Downward API.

To configure additional fields on the container logs, add the following section to your values file, and update your installation:

sources:
  containers:
    staticFields:
      FieldName: Value

To propagate Kubernetes Downward API information this way, you can also add the following section, for example:

additionalEnv:
  - name: MY_NODE_NAME
    valueFrom:
      fieldRef:
        fieldPath: spec.nodeName

Which will add an environment variable to the container MY_NODE_NAME, which can be configured as a static field value:

sources:
  containers:
    staticFields:
      NodeName: "${MY_NODE_NAME}"

The following issues have been noted when using the helm chart with Falcon CWP and can be safely ignored.