When constructing the HumioCluster resource, it is possible to leverage Kubernetes features like pod topology spread constraints, pod affinity and antiAffinity and taints and tolerations. These things can be configured on a per node pool level. These should be configured in a way so that pods are placed on worker nodes where pods are spread out uniformly across availability zones. So if we have a node pool with nodeCount 9 and worker nodes in 3 availability zones, the goal should be to place 3 pods in each availability zone.

It is possible to configure a node pool per availability zone, and use affinity and/or tolerations so pods for a specific node pool are placed on worker nodes in a specific availability zone. However, in most cases it is easier to have one node pool scheduled to multiple availability zones. To make sure LogScale pods knows the correct availability zone, we leverage an init container in the cluster pods. This init container gets the Kubernetes worker node name and looks up the availability zone using well-known labels on the worker node resource in the Kubernetes cluster. The zone information collected by the init container gets automatically passed to the LogScale pod by configuring the ZONE configuration option for the LogScale pod.

With zone information available to LogScale, it means LogScale will use this zone when making decisions, e.g. how to configure digest partitions and where to place segments.