Instance Sizing
This page gives an overview of typical Azure reference architecture sizing requirements. These are only a guide based on daily ingestion rates. It is important to monitor your cluster and make adjustments as required for your use case.
Note
Full details of exact sizing requirements for node types can be found in the Terraform Template File.
The following tables show suggested sizing information for the various node types in your cluster, based on ingestion rate:
X-Small - 1 TB/Day Ingestion
| Node Type | Desired Node Count | CPU Count | Instance Type | Memory | Storage |
|---|---|---|---|---|---|
| System Node | 2 | N/A | Standard_F4s_v2 | N/A | 40 GB |
| Kafka | 3 | 6 | Standard_E8s_v5 | 56 GB | 2 x 512 GB - Managed Premium |
| Digest Node | 3 | 6 | Standard_L8s_v3 | 48 GB | 1500 GB |
| Ingest Node | 3 | 6 | Standard_F8s_v2 | 10 GB | 60 GB |
| Ingress Node | 2 | 2 | Standard_F4s_v2 | 4 GB | 64 GB |
| UI Node | 2 | 3 | Standard_E4_v5 | 24 GB | 60 GB |
Small - 3 TB/Day Ingestion
| Node Type | Desired Node Count | CPU Count | Instance Type | Memory | Storage |
|---|---|---|---|---|---|
| System Node | 3 | N/A | Standard_F4s_v2 | N/A | 40 GB |
| Kafka | 5 | 14 | Standard_E16s_v5 | 102 GB | 3 x 1024 GB - Managed Premium |
| Digest Node | 6 | 14 | Standard_L16s_v3 | 100 GB | 3000 GB |
| Ingest Node | 6 | 6 | Standard_F8s_v2 | 10 GB | 60 GB |
| Ingress Node | 3 | 2 | Standard_F4s_v2 | 4 GB | 64 GB |
| UI Node | 3 | 3 | Standard_E4_v5 | 24 GB | 128 GB |
Medium - 5 TB/Day Ingestion
| Node Type | Desired Node Count | CPU Count | Instance Type | Memory | Storage |
|---|---|---|---|---|---|
| System Node | 3 | N/A | Standard_F4s_v2 | N/A | 40 GB |
| Kafka | 15 | 14 | Standard_E16s_v5 | 102 GB | 4 x 1024 GB - Managed Premium |
| Digest Node | 21 | 14 | Standard_L16s_v3 | 102 GB | 1024 GB |
| Ingest Node | 15 | 14 | Standard_F16s_v2 | 28 GB | 60 GB |
| Ingress Node | 6 | 6 | Standard_F8s_v2 | 10 GB | 128 GB |
| UI Node | 6 | 7 | Standard_E8_v5 | 56 GB | 250 GB |
Large - 10 TB/Day Ingestion
| Node Type | Desired Node Count | CPU Count | Instance Type | Memory | Storage |
|---|---|---|---|---|---|
| System Node | 6 | N/A | Standard_F8s_v2 | N/A | 40 GB |
| Kafka | 30 | 18 | Standard_E20s_v5 | 140 GB | 8 x 2048 GB - Managed Premium |
| Digest Node | 42 | 30 | Standard_L32s_v3 | 240 GB | 6000 GB |
| Ingest Node | 15 | 30 | Standard_F32s_v2 | 58 GB | 128 GB |
| Ingress Node | 6 | 14 | Standard_F16s_v2 | 22 GB | 128 GB |
| UI Node | 9 | 14 | Standard_E16_v5 | 102 GB | 250 GB |
X-Large - 30 TB/Day Ingestion
| Node Type | Desired Node Count | CPU Count | Instance Type | Memory | Storage |
|---|---|---|---|---|---|
| System Node | 6 | N/A | Standard_F8s_v2 | N/A | 40 GB |
| Kafka | 33 | 30 | Standard_E32s_v5 | 240 GB | 10 x 2048 GB - Managed Premium |
| Digest Node | 78 | 62 | Standard_L64s_v3 | 500 GB | 12000 GB |
| Ingest Node | 18 | 46 | Standard_F48s_v2 | 92 GB | 128 GB |
| Ingress Node | 18 | 14 | Standard_F16s_v2 | 22 GB | 128 GB |
| UI Node | 18 | 18 | Standard_E20_v5 | 148 GB | 500 GB |