Digest Node Storage

A digest node is responsible for constructing segment files for incoming events. Segment files are the internal storage format in LogScale.

Digest nodes are also involved in executing the real-time searches of data, and the more recently received events. Queries on older events are handled by storage nodes.

Once a segment file is constructed it is passed on to a storage node.

When sizing storage requirements for digest nodes, you need to take into account:

  • Daily Data ingest volume

  • Retention period (typically 24 to 48 hours)

  • Compression ratio (from raw data to segments)

For example:

  • 100 GB daily ingest

  • 48 hours retention

  • 100 GB x 2 days = 200 GB raw data

  • 200 GB x 0.02 = 4 GB pre-compression

  • With 3:1 compression: 4 GB ÷ 3 = ~1.33 GB

  • With 20% buffer: 1.33 GB × 1.2 = ~1.6 GB would be the minimum recommended