AWS Reference Architecture

This section of the documentation contains Terraform configurations to deploy an AWS-based architecture for LogScale.

Important

If you are using an older version of the Reference Architecture, then you can still read the now deprecated material in the following Knowledgebase Articles:

Note

The CrowdStrike AWS Reference Architecture automations can be found in the GitHub repository.

Use Cases

The logscale-aws Terraform modules provide the following architecture choices.

Basic use cases:

  • Development, Testing

  • Smaller Search Teams

  • Minimal ingest processing

Ingress use cases:

  • Ingress tier in specified DMZ

  • Ingress resources not shared with Kubernetes system resources

Dedicated UI use cases:

  • Separated ingress tier

  • Separated UI tier

  • Ingest/Digest on same hosts

Advanced use cases:

  • Dedicated processing tier (ingestion) scaling separate from digest/storage

  • Dedicated UI tier for dashboards and search

  • Segmentation of system responsibilities and independent scaling

Components

All architecture choices rely on the same underlying technologies:

  • AWS Elastic Kubernetes Service (EKS)

  • AWS Load Balancer - Allow data ingest and UI access to the environment

  • AWS NAT Gateway - Allows egress data to pull images for Kubernetes pods

  • AWS SSM - Enables access to the worker nodes via the AWS management console

  • AWS VPC - Provides private, public, and intra subnets across three availability zones

  • AWS MSK - Sets up a managed Kafka cluster within AWS

  • AWS IAM - Manages roles and policies for EKS and associated services

  • AWS S3 - Object storage for LogScale data durability

The following Kubernetes Apps are also used:

  • cert-manager: For automated provisioning of certificates in the environment

  • humio-operator: For provisioning of LogScale clusters in the environment

  • nginx-ingress: For connecting the AWS Load Balancers

  • ExternalDNS: For synchronizing exposed Kubernetes Services and Ingresses with Route53

  • strimzi-operator (optional): For provisioning Kafka broker nodes (kraft mode)

Diagnostic (Audit) Logging

Logging to S3 is enabled by default for the following resources:

  • AWS MSK

  • AWS Elastic Kubernetes Service

  • AWS S3

The following guidance box provides links to the main section of this documentation:

Architecture Types

This section describes the supported architecture types.

Requirements

This section describes prerequisites for deploying a AWS reference architecture for LogScale.

Sizing

This section describes suggested sizing for deploying a AWS reference architecture for LogScale.

Terraform Modules

This section describes Terraform modules for deploying the AWS reference architecture.

Build Process

This section describes the build process.

Maintaining

This section describes maintaining, upgrading, and scaling the reference architecture implementation.