Digest Node Storage

A digest node is responsible for constructing segment files for incoming events. Segment files are the internal storage format in LogScale.

Digest nodes are also involved in executing the real-time searches of data, and the more recently received events. Queries on older events are handled by storage nodes.

Once a segment file is constructed it is passed on to a storage node.

When sizing storage requirements for digest nodes, you need to take into account:

  • Daily Data ingest volume
  • Retention period (typically 24 to 48 hours)
  • Compression ratio (from raw data to segments)

For example:

  • 100 GB daily ingest
  • 48 hours retention
  • 100 GB x 2 days = 200 GB raw data
  • 200 GB x 0.02 = 4 GB pre-compression
  • With 3:1 compression: 4 GB ÷ 3 = ~1.33 GB
  • With 20% buffer: 1.33 GB × 1.2 = ~1.6 GB would be the minimum recommended