Instance Sizing

Assumptions:

  • 30 Day Retention on NVME (gen4)

  • 20% Overhead left on NVME (gen4)

  • 10x Compression

  • Supported Object Storage provider

  • Kafka 5x Compression - 24 Hour Storage

  • LogScale requires a kafka cluster most commonly provided by the Strimzi Kubernetes operator or Amazon MSK (required up to LogScale 1.107.0).

Kafka clusters are separate from LogScale clusters to avoid resource contention and allow independent management.

Important

Any deployed LogScale instance should not be sharing resources with any other services and it should be a dedicated node to avoid resource contention. LogScale assumes that all spare memory is available for use by LogScale.

X-Small - 1 TB/Day Ingestion
Software Instances vCPU Memory Storage Total Storage
LogScale 3 16 64 GB NVME 2 TB 6 TB
Kafka 3 4 8 GB SSD 500 GB 1.5 TB
ZooKeeper (up to LogScale 1.107) 3 4 8 GB SSD 50 GB 150 GB
KRaft controller (after LogScale 1.107) 3 4 16 GB SSD 64 GB 150 GB
Small - 3 TB/Day Ingestion
Software Instances vCPU Memory Storage Total Storage
LogScale 3 32 128 GB NVME 6 TB 18 TB
Kafka 3 4 8 GB SSD 1 TB 3 TB
ZooKeeper (up to LogScale 1.107) 3 4 8 GB SSD 50 GB 150 GB
KRaft controller (after LogScale 1.107) 3 4 16 GB SSD 64 GB 150 GB
Medium - 5 TB/Day Ingestion
Software Instances vCPU Memory Storage Total Storage
LogScale 6 32 128 GB NVME 6 TB 36 TB
Kafka 3 8 16 GB SSD 1 TB 3 TB
ZooKeeper (up to LogScale 1.107) 3 4 8 GB SSD 50 GB 150 GB
KRaft controller (after LogScale 1.107) 3 4 16 GB SSD 64 GB 150 GB
Large - 10 TB/Day Ingestion
Software Instances vCPU Memory Storage Total Storage
LogScale 12 32 128 GB NVME 6 TB 72 TB
Kafka 6 8 16 GB SSD 1 TB 6 TB
ZooKeeper (up to LogScale 1.107) 3 4 8 GB SSD 50 GB 150 GB
KRaft controller (after LogScale 1.107) 3 4 16 GB SSD 64 GB 150 GB
X-Large - 30 TB/Day Ingestion
Software Instances vCPU Memory Storage Total Storage
LogScale 30 32 128 GB NVME 7 TB 210 TB
Kafka 6 8 16 GB SSD 1.5 TB 13.5 TB
ZooKeeper (up to LogScale 1.107) 3 4 8GB SSD 50 GB 150 GB
KRaft controller (after LogScale 1.107) 3 4 16 GB SSD 64 GB 150 GB