Collect Kubernetes Pod Logs

Kubernetes is a portable, extensible, open source platform for managing containerized workloads and services.

Kubernetes runs your workload by placing containers into pods to run on nodes. A node may be a virtual or physical machine, depending on the cluster.

  • A node is a worker machine in Kubernetes and may be either a virtual or a physical machine, depending on the cluster.

  • A pod is a Kubernetes abstraction that represents a group of one or more application containers (such as Docker), and some shared resources for those containers. A Pod models an application-specific "logical host" and can contain different application containers which are relatively tightly coupled.

  • A container image is a ready-to-run software package containing everything needed to run an application: the code and any runtime it requires, application and system libraries, and default values for any essential settings.

The following describes:

Deploy Falcon LogScale Collector for Log Forwarding

When it comes to managing micro-services in a Kubernetes cluster, LogScale is a great way to get insights into application status.

The Falcon LogScale Collector can be deployed in a Kubernetes to forwarding log messages from the applications deployed in the cluster.

In case of e.g. an application crashing on a virtual machine, the logs from the application are still available until deleted. In Kubernetes, when pods crash, are deleted or scheduled on a new node, the logs from the application containers are lost. For this reason to gain insight into e.g. why a crash occurred, you need the logs forwarded to a centralized log management solution like e.g. LogScale.

Several different deployments are possible, but the model below describes node-level logging using the DaemonSet model. (This is delivered as an out-of-the-box solution consisting of a Log Collector Helm chart and container image) DaemonSet approach in which a node-level Falcon LogScale Collector runs on every node, and handles logging for all application containers in pods on the node.

Node level logging overview

Figure 43. Node Level Logging


Node Level Logging

In this scenario the Falcon LogScale Collector is deployed as a DaemonSet on a Kubernetes node to ingest logs from applications running in pods on that node.

The Falcon LogScale Collector is deployed as a Kubernetes DaemonSet, which is a Kubernetes feature that lets you run a Kubernetes pod on all cluster nodes that meet certain criteria. Every time a new node is added to a cluster, the pod is added to it, and when a node is removed from the cluster, the pod is removed.

Node-level logging creates one Falcon LogScale Collector per node and does not require any changes to the applications running on the node.

Containers write to stdout and stderr, but with no agreed format. A node-level Falcon LogScale Collector collects these logs and forwards them in real-time to LogScale for live analysis and storage/future analysis.

This is accomplished by running the Falcon LogScale Collector in a container that has access to a directory with log files from all of the application containers in all pods on that node.

CrowdStrike provides a Falcon LogScale Collector Helm chart for deploying the Falcon LogScale Collector in Kubernetes as a DaemonSet, collecting logs from pods.