Skip to content
LogScale Self-Hosted Deployment
This is an archived version of the documentation downloaded from LogScale Documentation Archives. Up to date documentation, and documentation for later versions, is available on LogScale Documentation

## Azure Reference Architecture

This section of the documentation contains Terraform configurations to deploy a Microsoft Azure-based architecture for LogScale. It leverages multiple Azure services including Azure Key Vault, Azure Kubernetes Service, and Azure Storage.

The following guidance box provides links to the main section of this documentation:

> Architecture Types

> This section describes the supported architecture types.

> Requirements

> This section describes prerequisites for deploying a Microsoft Azure reference architecture for LogScale.

> Terraform Modules

> This section describes Terraform modules for deploying the Azure reference architecture.

> Build Process

> This section describes the build process.

### Overview

The `logscale-azure` terraform modules provide the following architecture choices:

Basic use cases:

- Development, Testing

- Smaller Search Teams

- Minimal ingest processing

Ingress use cases:

- Ingress tier in specified DMZ

- Ingress resources not shared with Kubernetes system resources

Dedicated UI use cases:

- Separated ingress tier

- Separated UI tier

- Ingest/Digest on same hosts

Advanced use cases:

- Dedicated processing tier (ingestion) scaling separate from digest/storage

- Dedicated UI tier for dashboards and search

- Segmentation of system responsibilities and independent scaling

All architecture choices rely on the same underlying technologies:

- Azure Infrastructure and Resource Groups - Note that the user will have 2 resource groups, 1 created by this terraform and the other associated to the managed Kubernetes services.

- Azure Kubernetes Service (AKS)

- Azure NAT Gateway - Allows egress data to pull images for kubernetes pods

- Azure KeyVault:

  - Stores kube secrets for the environment

  - Stores LogScale encryption string

  - Stores LogScale single user password

  - Stores encryption key for AKS disk encryption set

- Azure Load Balancer - Allow data ingest and UI access to the environment

- Azure Storage Account - Object storage for LogScale data durability

- Kubernetes Apps:

  - cert-manager: For automated provisioning of certificates in the environment

  - strimzi-operator: For provisioning Kafka broker nodes (kraft mode)

  - humio-operator: For provisioning of LogScale clusters in the environment

  - nginx-ingress: For connecting the Azure Load Balancer

By default:

- Kubernetes API is public but has IP restrictions applied based on user-controlled variables to limit access.

- Ingest endpoint is publicly available:

  - Port 80: Used for Let's Encrypt certificate signing - Access is global but only available temporarily during ACME challenge/response while the cert-manager response pod is alive.

  - Port 443: Used for ingest/UI to LogScale - Access is limited to user configured IP list variable

- Storage account access is restricted to:

  - User provided IP ranges

  - Subnet(s) created for LogScale nodes

  - Azure Trusted Services, Logging, and Metrics

- KeyVault access is restricted to:

  - User provided IP ranges

  - Azure Trusted Services

- All AKS nodes are made available on required ports to AzureCloud IP ranges to allow for Kubernetes control plane operations

- All AKS nodes are made accessible within the VNET

### Important

Some settings are configurable and still pending testing. For example, it is a configuration option to make the Load Balancer and Kubernetes API internal-only but this has not been tested.

Networking is created as follows:

Purpose | Architecture Created | IP Range
--- | --- | ---
Virtual Network | all | 172.16.0.0/16
Kubernetes System Nodes | all | 172.16.0.0/24
Kafka Nodes | all | 172.16.2.0/24
Bastion Nodes | all | 172.16.1.0/26
LogScale UI Nodes | advanced | 172.16.6.0/24
Nginx Ingress Nodes | ingress, advanced | 172.16.4.0/24
LogScale Ingest Nodes | advanced | 172.16.5.0/24
LogScale Digest Nodes | all | 172.16.3.0/24

### Diagnostic (Audit) Logging

Logging can be enabled for supported resources to supported target destinations that must be created outside of this terraform. Currently supported resources:

- Azure Key Vault

- Azure Kubernetes Service

- Azure Storage Account

Currently supported destinations:

- Azure Event Hub

- Azure Storage Account

- Azure Log Analytics

Update your created `.tfvars` file to enable this feature. Example:

ini

    enable_auditlogging_to_storage      = true
    diag_logging_storage_account_id     = "/subscriptions/$mysubscription/resourceGroups/$myresourcegroup/providers/Microsoft.Storage/storageAccounts/$mystorageacct"

#### Enter search term