Cross-Region Storage Access

During DR recovery, the secondary cluster must read the global snapshot from the primary cluster's Azure Blob Storage container.

Requirements:

  • Network-level access - Storage firewall IP rules allowing the secondary's NAT Gateway IP

  • Authentication - Storage account key (AZURE_RECOVER_FROM_ACCOUNTKEY)

  • (Optional) RBAC - Storage Blob Data Reader role on primary storage account

Storage Container Naming

Storage account names must be globally unique in Azure. This repo intentionally includes a short random prefix (random_string.name-modifier) in local.resource_name_prefix, so the exact storage account/container names are:

  • Stable within a state file (the random prefix is stored in Terraform state)

  • Not knowable in advance before the first apply

For DR operations, do not guess names. Use Terraform outputs in each state:

shell
terraform output -raw storage_acct_name
terraform output -raw storage_acct_container_name
terraform output -raw storage_acct_blob_endpoint
terraform output -raw AZURE_STORAGE_OBJECT_KEY_PREFIX  # blob prefix for global snapshots
Cluster Container Name Example
Primary terraform output -raw storage_acct_container_name
Secondary terraform output -raw storage_acct_container_name

AZURE_STORAGE_OBJECT_KEY_PREFIX is the blob name prefix under which LogScale stores global snapshots and data segments (e.g., z072ef-drpriglobalsnapshots/). It is derived from local.resource_name_prefix and is used by AZURE_RECOVER_FROM_OBJECT_KEY_PREFIX to tell the secondary cluster where to find the primary's snapshots. The Op Guide's snapshot deletion steps depend on this value.

Encryption Key Architecture

The main Encryption Key requirements are listed here:

  • Primary generates the encryption key via the pre-install module on first deploy

  • Secondary receives the key automatically via terraform_remote_state (primary mechanism). The existing_storage_encryption_key variable is a fallback for environments where remote state is unavailable

  • Keys are stored as Kubernetes secrets (logscale-storage-encryption-key), never committed to version control

  • Same key must be used across both clusters for DR recovery

  • The encryption key is passed to LogScale via AZURE_STORAGE_ENCRYPTION_KEY environment variable (secretKeyRef)

Data Flow Summary
Direction Data Exchanged Purpose
Primary โ†’ Secondary storage_account_id Target for firewall update
Primary โ†’ Secondary storage_encryption_key Decrypt global snapshot
Primary โ†’ Secondary storage_account_key Authenticate to primary storage
Primary โ†’ Secondary storage_account_name, storage_container_name Locate primary bucket
Primary โ†’ Secondary storage_ip_rules Existing IP rules to merge
Primary โ†’ Secondary storage_vnet_rules Existing VNet rules to merge
Primary โ†’ Secondary AZURE_STORAGE_OBJECT_KEY_PREFIX Snapshot path prefix to locate data in primary bucket
Primary โ†’ Secondary traffic_manager_primary_endpoint_id Failover function alert target (standby monitors this endpoint)
Secondary โ†’ Primary AKS outbound IPs Merged into primary storage firewall (for DR recovery)
Secondary โ†’ Primary Subnet IDs Merged into primary VNet rules (for DR recovery)
Primary โ†’ Secondary AKS outbound IPs Merged into secondary storage firewall (for reverse recovery / failback)
Primary โ†’ Secondary Subnet IDs Merged into secondary VNet rules (for reverse recovery / failback)
Verification
shell
# Check primary storage account firewall IP rules
az storage account show \
  --name <primary-storage-account> \
  --resource-group <primary-rg> \
  --query "networkRuleSet.ipRules" -o table
# Get secondary AKS outbound IPs for comparison
terraform workspace select secondary
terraform output aks_effective_outbound_ips
# Test connectivity from secondary LogScale pod
kubectl exec -n logging -it <humio-pod> --context aks-secondary -- \
  curl -s -o /dev/null -w "%{http_code}" \
  "https://<primary-storage-account>.blob.core.windows.net/<container>?restype=container"
# Confirm secondary subnets in primary storage VNet rules
az storage account show \
  --name <primary-storage-account> \
  --resource-group <primary-rg> \
  --query "networkRuleSet.virtualNetworkRules[].virtualNetworkResourceId" -o tsv
# Confirm Storage Blob Data Reader RBAC assignment
az role assignment list --scope "/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Storage/storageAccounts/<sa>" \
  --query "[?roleDefinitionName=='Storage Blob Data Reader']" -o table