Multiple namespaces are created in Kubernetes during the terraform application process in order to promote security and separation of the applications. All namespaces are created using variable var.k8s_namespace_prefix (default: log). Assuming the default value for k8s_namespace_prefix, Terraform creates the following namespaces in kubernetes:

The cluster_size.tpl file specifies the available parameters for different sizes of LogScale clusters. This template defines various cluster sizes, for example xsmall, small, medium and their associated configurations, including node counts, instance types, disk sizes, and resource limits. The Terraform configuration uses this template to dynamically configure the LogScale deployment based on the selected cluster size.

The data from cluster_size.tpl is retrieved and rendered by the locals.tf file. The locals.tf file uses the jsondecode function to parse the template and select the appropriate cluster size configuration based on the logscale_cluster_size variable.

Example:

# Local Variables
  locals {
    # Render a template of available cluster sizes
    cluster_size_template = jsondecode(templatefile("${path.module}/cluster_size.tpl", {}))
    cluster_size_rendered = {
      for key in keys(local.cluster_size_template) :
      key => local.cluster_size_template[key]
    }
    cluster_size_selected = local.cluster_size_rendered[var.logscale_cluster_size]
  }

LogScale will be configured with a default set of configuration values that can be overridden or added to by defining var.user_logscale_envars in your TFVAR_FILE. For example, to change default values for LOCAL_STORAGE_MIN_AGE_DAYS and LOCAL_STORAGE_PERCENTAGE, you can set this in your TFVAR_FILE, as shown in the following example:

user_logscale_envvars = [ { "name" = "LOCAL_STORAGE_MIN_AGE_DAYS", "value" = "7" }, { "name" = "LOCAL_STORAGE_PERCENTAGE", "value" = "85" } ]

This mechanism also supports referencing Kubernetes secrets should you provision them outside this Terraform:

user_logscale_envvars = [
        {
        "name" = "SECRET_LOGSCALE_CONFIGURATION",
        "valueFrom" = {
            "secretKeyRef" = {
                "key"  = "secret_value"
                "name" = "kubernetes_secret_name"
            }
        }
        },
        { "name" = "LOCAL_STORAGE_MIN_AGE_DAYS", "value" = "7" },
        { "name" = "LOCAL_STORAGE_PERCENTAGE", "value" = "85" }
    ]

The default environment values set by this Terraform are as follows:

Values marked with * are removed when var.provision_kafka_servers is set to false.

If Kafka already exists and meets the following expectations, it can be used in place of Strimzi created by this Terraform. Expected configuration:

In order to use your own Kafka, make the following modifications to the execution instructions:

By default, certificates will be generated and placed on the ingress. You can bring your own certificate to the ingress by:

When leveraging this Terraform repository, you must run terraform using the -target flag to apply specific modules. The latter half of the terraforming process requires access to a Kubernetes API to successfully plan and apply changes.

After the environment is fully built, the targeted approach isn't strictly required but remains recommended to ensure correct order of operations.