Configure a LogScale Collector Helm Chart

The LogScale Collector Helm Chart simplifies the ingest of Pod logs in LogScale. The basic configuration consists of only two values that must be set:

humioAddress: https://<your logscale cluster> # insert the cluster URL here.
humioIngestTokenSecretName: logscale-collector-token # insert a secret name that contains the ingestToken.

The configuration options can be set via the --set argument, or provided via a YAML file -f values.yaml.

Example Helm Chart Configuration

This is an example Helm Configuration for LogScale Collector.

  1. Create a Secret.

    kubectl create secret generic logscale-collector-token --from-literal=ingestToken="INSERT INGEST TOKEN HERE"
  2. Add the Helm Repository.

    kubectl create secret generic logscale-collector-token --from-literal=ingestToken="INSERT INGEST TOKEN HERE"
  3. Install helm via:

    • values.yaml file.

      helm install -g logscale-collector-helm/logscale-collector --values values.yaml

      Using the argument -g generates a unique name, which can be substituted -g with the desired installation name.

    • command line arguments

      helm install -g logscale-collector-helm/logscale-collector --set humioAddress=https://logscale.cluster,humioIngestTokenSecretName=logscale-collector-token

      Using the argument -g generates a unique name, which can be substituted -g with the desired installation name.

Optionally, the Helm chart can be managed through terraform. Example terraform file:

locals {
  ingestToken  = "your ingest token here"
  humioAddress = "https://logscale.cluster"
}

provider "helm" {
  kubernetes {
    config_path = "~/.kube/config"
  }
}

provider "kubernetes" {
  config_path = "~/.kube/config"
}

resource "kubernetes_secret" "logscale_collector_token" {
  metadata {
    name = "logscale-collector-token"
  }

  data = {
    ingestToken = local.ingestToken
  }
}


resource "helm_release" "logscale_collector" {
  name = "logscale-collector"

  repository = "https://registry.crowdstrike.com/log-collector-us1-prod"
  chart      = "logscale-collector"

  set {
    name  = "humioAddress"
    value = local.humioAddress
  }

  set {
    name  = "humioIngestTokenSecretName"
    value = kubernetes_secret.logscale_collector_token.metadata[0].name
  }
}
Advanced Helm Configuration Options

The Helm chart contains additional options that can be configured. A dot `.` in the key describes a nested field.

Key Type Default Description
image string   *The latest container image that is released* | A container registry URL to the container image to run.
imagePullPolicy string 'always' Controls when Kubernetes pulls the container image.
imagePullSecrets string   If the container image is hosted on a private registry, credentials might be needed.
humioAddress string Uses `humioAddress` by default. Address to the LogScale Cluster where logs will be ingested.
logLevel string warn The log level of the LogScale Collector. This controls what the Pod logs.
humioIngestTokenSecretName string 'ingestToken' Name of the Kubernetes secret that contains the ingest token for Pod logs.
humioIngestTokenSecretKey string   Name of the key in the Kubernetes secret referenced by `humioIngestTokenSecretKey`.
humioDebugTokenSecretName string   Name of a Kubernetes secret that contains an ingest token for debug logs. See debug logging.
humioDebugTokenSecretKey string 'debugToken' Name of the key in the Kubernetes secret referenced by `humioDebugTokenSecretName`.
queueMemoryLimitMB integer 8mb Size of the in-memory queue used by the LogScale Collector.
queueFlushTimeoutMillis integer '100' Flush interval of the in-memory queue used by the LogScale Collector.
collectJournal bool   The LogScale Collector Helm chart collects host logs from Journal if set to true.
fleetManagement.url string   Address to the LogScale Cluster where the LogScale Collector should enroll to fleet management.
fleetManagement.enrollmentTokenSecretName string   Name of the Kubernetes secret that contains the enrollment token for fleet management.
fleetManagement.enrollmentTokenSecretKey string   Name of the key in the Kubernetes secret referenced by `fleetManagement.enrollmentTokenSecretName`.
fleetManagement.ephemeralTimeoutHours integer 2 When enrolled in fleet management, the LogScale Collector will be automatically unenrolled after it has been offline for this duration.
resources Resources 2 CPUs and 1Gi memory. Set Kubernetes resources requests and limits for the Pod.
sources.containers.staticFields JSON object   The keys and values of the JSON object are added as fields to the ingested Pod logs.
sources.journal.staticFields JSON object   The keys and values of the JSON object are added as fields to the ingested Journal logs.
runOnControlPlane bool 'false' Set to true to add tolerations to the DaemonSet that allows it to run on control plane nodes.
tolerations Tolerations   Add additional tolerations to the DaemonSet.
managedSecrets.ingestToken string   Insert an ingest token here to configure the Helm chart to managed a Kubernetes secret. Omit the `humioIngestTokenSecretName` key when using this.
managedSecrets.debugToken string   Insert an debug ingest token here to configure the Helm chart to managed a Kubernetes secret. Omit the `humioDebugTokenSecretName` key when using this.
managedSecrets.enrollmentToken string   Insert an enrollment token here to configure the Helm chart to managed a Kubernetes secret. Omit the `fleetManagement.enrollmentTokenSecretName` key when using this.
additionalSinkOptions Sinks ( sinks)   Add additional LogScale Collector sink options. The keys are inserted directly into the sink specification
additionalEnv Environment variables   Add additional Kubernetes environment variable spec to the Pod spec.
extraVolumes Volumes   Add additional Kubernetes `volumes` to the Pod spec.
extraVolumeMounts Volume mounts   Add additional Kubernetes `volumeMounts` to the container spec.
nodeSelector Node selector   Add `nodeSelector` spec to the Pod spec.
additionalSources Sources   Add additional LogScale Collector sources to the config.
additionalSinks Sinks   Add additional LogScale Collector sinks to the config.
Enroll in Fleet Management

The Helm chart supports enrolling the LogScale Collector into fleet management, which enables the LogScale Collector to send additional metrics to LogScale. The LogScale Collector is displayed on fleet overview, along with details on its health status. By default the LogScale Collector will be un-enrolled after being offline for 2 hours. (Configurable via fleetManagement.ephemeralTimeoutHours).

Setup with a Secret

Create a secret, or add an additional field to the existing secret.

kubectl create secret generic logscale-collector-enrollment-token --from-literal=enrollmentToken="INSERT ENROLLMENT TOKEN HERE"

Add the values to the Helm chart:

fleetManagement: enrollmentTokenSecretName: logscale-collector-enrollment-token
Setup with a Managed Secret

Enter the enrollment token directly into the values of the Helm chart.

managedSecrets:
  enrollmentToken: "INSERT ENROLLMENT TOKEN HERE"
Add Additional Fields to Ingested Logs

The Helm chart can configure a static_fields transform on the log sources to add additional fields to the ingested logs. This can be useful to add context from the Kubernetes cluster.

Static Fields
sources:
  containers:
    staticFields:
      # Add any key and value you need. The key and value must be a strings.
      cluster: "kubernetes.local"
Add Kubernetes Downward API to the ingested logs

Combine the staticFields element with additionalEnv, to add information from the Downward API.

sources:
  containers:
    staticFields:
      # Add any key and value you need. The key and value must be a strings.
      node: "${MY_NODE_NAME}"

additionalEnv:
  - name: MY_NODE_NAME
    valueFrom:
      fieldRef:
        fieldPath: spec.nodeName
Set Custom Sink Options

The sink can be configured by adding additional sink options which alter; the proxy setting, compression level or specify TLS options.

  • Setting a proxy

    additionalSinkOptions:
      proxy: https://user:pass@proxy:port
  • Setting the compression level

    additionalSinkOptions:
      compression: gzip
      compressionLevel: 1
  • Setting a custom CA certificate:

    additionalSinkOptions:
      tls:
        caCert: |
          -----BEGIN CERTIFICATE-----
          ...
          -----END CERTIFICATE-----