Before starting the deployment, ensure you have the following tools and access:

It is additionally recommended, but not required, to install Helm 3.17.0 or later, for troubleshooting Helm-based Kubernetes deployments.

The account running this Terraform needs to be assigned the Owner role for the target subscription due to the assignment of roles to the managed identity used by the kubernetes control plane. Role assignment for the Kubernetes cluster is as follows:

There are four variables that control public access to the environment. You set these in your Terraform configuration file, set by the TFVAR_FILE variable, as shown in the following example:

ip_ranges_allowed_to_kubeapi            = ["192.168.3.32/32", "192.168.4.1/32"]
ip_ranges_allowed_https                 = ["192.168.1.0/24"]
ip_ranges_allowed_to_bastion            = ["192.168.3.32/32", "192.168.4.1/32"]
ip_ranges_allowed_kv_access             = ["192.168.3.32/32", "192.168.4.1/32"]

Multiple namespaces are created in Kubernetes during the terraform application process in order to promote security and separation of the applications. All namespaces are created using variable var.k8s_namespace_prefix (default: log). Assuming the default value for k8s_namespace_prefix, terraform creates the following namespaces in kubernetes:

The cluster_size.tpl file specifies the available parameters for different sizes of LogScale clusters. This template defines various cluster sizes, for example xsmall, small, medium and their associated configurations, including node counts, instance types, disk sizes, and resource limits. The Terraform configuration uses this template to dynamically configure the LogScale deployment based on the selected cluster size.

The data from cluster_size.tpl is retrieved and rendered by the locals.tf file. The locals.tf file uses the jsondecode function to parse the template and select the appropriate cluster size configuration based on the logscale_cluster_size variable.

Example:

# Local Variables
  locals {
    # Render a template of available cluster sizes
    cluster_size_template = jsondecode(templatefile("${path.module}/cluster_size.tpl", {}))
    cluster_size_rendered = {
      for key in keys(local.cluster_size_template) :
      key => local.cluster_size_template[key]
    }
    cluster_size_selected = local.cluster_size_rendered[var.logscale_cluster_size]
  }

LogScale will be configured with a default set of configuration values that can be overridden or added to by defining var.user_logscale_envars in your TFVAR_FILE. For example, to change default values for LOCAL_STORAGE_MIN_AGE_DAYS and LOCAL_STORAGE_PERCENTAGE, you can set this in your TFVAR_FILE, as shown in the following example:

user_logscale_envvars = [ { "name" = "LOCAL_STORAGE_MIN_AGE_DAYS", "value" = "7" }, { "name" = "LOCAL_STORAGE_PERCENTAGE", "value" = "85" } ]

This mechanism also supports referencing Kubernetes secrets should you provision them outside this Terraform:

user_logscale_envvars = [
        {
        "name" = "SECRET_LOGSCALE_CONFIGURATION",
        "valueFrom" = {
            "secretKeyRef" = {
                "key"  = "secret_value"
                "name" = "kubernetes_secret_name"
            }
        }
        },
        { "name" = "LOCAL_STORAGE_MIN_AGE_DAYS", "value" = "7" },
        { "name" = "LOCAL_STORAGE_PERCENTAGE", "value" = "85" }
    ]

The default environment values set by this Terraform are as follows:

Values marked with * are removed when var.provision_kafka_servers is set to false.

If Kafka already exists and meets the following expectations, it can be used in place of Strimzi created by this Terraform. Expected configuration:

In order to use your own Kafka, make the following modifications to the execution instructions:

By default, a Let's Encrypt certificate will be generated and placed on the ingress controller. You can bring your own certificate to the ingress by:

module "logscale" {
    source                                        = "./modules/kubernetes/logscale"
    <truncated for readability>
    
    use_custom_certificate                        = true
    custom_tls_certificate_keyvault_entry         = "my-keyvault-item-name"
    
    <truncated for readability>
    }

A module exists in this Terraform that allows for provisioning a certificate via Azure Key Vault. This is self-signed by default but the module allows for alternative certificate issuers depending on your environment.

module "azure-selfsigned-cert" {
    source                                        = "./modules/azure/certificate"
    azure_keyvault_id                             = module.azure-keyvault.keyvault_id
    logscale_public_fqdn                          = "this-is-a-test.local"
    name_prefix                                   = local.resource_name_prefix
    subject_alternative_names                     = [module.azure-core.ingress-pub-fqdn, "othername.local"]  
    cert_issuer                                   = "Self"    
    }

When leveraging this Terraform repository, you must run terraform using the -target flag to apply specific modules. The latter half of the terraforming process requires access to a Kubernetes API to successfully plan and apply changes.

After the environment is fully built, the targeted approach isn't strictly required but remains recommended to ensure proper order of operations.

A module is provided to build a bastion host. This host will have a public IP attached to it for SSH access and can be used for operations in the environment when setting up private kubernetes API access:

terraform apply -target module.bastion-host-1 -var-file $TFVAR_FILE