Multiple namespaces are created in Kubernetes during the terraform application process in order to promote security and separation of the applications. All namespaces are created using variable var.k8s_namespace_prefix (default: log). Assuming the default value for k8s_namespace_prefix, Terraform creates the following namespaces in kubernetes:

The cluster_size.tpl file specifies the available parameters for different sizes of LogScale clusters. This template defines various cluster sizes, for example xsmall, small, medium and their associated configurations, including node counts, instance types, disk sizes, and resource limits. The Terraform Configuration uses this template to dynamically configure the LogScale deployment based on the selected cluster size. See also Sizing.

The data from cluster_size.tpl is retrieved and rendered by the locals.tf file. The locals.tf file uses the jsondecode function to parse the template and select the appropriate cluster size configuration based on the logscale_cluster_size variable.

This is shown in the following example:

# Local Variables
  locals {
    # Render a template of available cluster sizes
    cluster_size_template = jsondecode(templatefile("${path.module}/cluster_size.tpl", {}))
    cluster_size_rendered = {
      for key in keys(local.cluster_size_template) :
      key => local.cluster_size_template[key]
    }
    cluster_size_selected = local.cluster_size_rendered[var.logscale_cluster_size]
  }

LogScale will be configured with a default set of configuration values that can be overridden or added to by defining var.user_logscale_envars in your TFVAR_FILE. For example, to change default values for LOCAL_STORAGE_MIN_AGE_DAYS and LOCAL_STORAGE_PERCENTAGE, you can set this in your TFVAR_FILE, as shown in the following example:

user_logscale_envvars = [
        {
        "name" = "SECRET_LOGSCALE_CONFIGURATION",
        "valueFrom" = {
            "secretKeyRef" = {
                "key"  = "secret_value"
                "name" = "kubernetes_secret_name"
            }
        }
        },
        { "name" = "LOCAL_STORAGE_MIN_AGE_DAYS", "value" = "7" },
        { "name" = "LOCAL_STORAGE_PERCENTAGE", "value" = "85" }
    ]

The default environment values set by this Terraform are as follows:

If Kafka already exists and meets the following expectations, it can be used in place of Strimzi created by this Terraform. Expected configuration:

In order to use your own Kafka, make the following modifications to the execution instructions:

By default, a Let's Encrypt certificate will be generated and placed on the ingress controller. You can bring your own certificate to the ingress by:

Importing or generating a certificate as a k8s secret under the name <resource_name_prefix>-tls-certificate Setting use_own_certificate_for_ingress to true in the .tfvars file.

When leveraging this Terraform repository, you must run terraform using the -target flag to apply specific modules. The latter half of the terraforming process requires access to a Kubernetes API to successfully plan and apply changes.

After the environment is fully built, the targeted approach isn't strictly required but remains recommended to ensure proper order of operations.