Query Capacity
When querying data, the following factors affect performance:
Number of digest nodes
The number of digest nodes affects the performance for real-time searches, and the execution of the historical part of searches on recent events.
Storage nodes
The number of storage nodes affects the performance for historic searches, particularly where the data has had to be reloaded from bucket storage.
Time range of the query
The time range of the query has an impact on the overall performance and capacity planning if queries often refer to older data that may have been sent to bucket storage. Executing that query requires recovery of the segment from bucket storage and then storage of the recovered data on the local disk of storage nodes.
Space on nodes for segment data
The space available for segment data on storage nodes and whether the segments are on primary, secondary, or bucket storage.
The execution of queries can be summarized with the following diagram:
Choosing whether to increase the number of digest or storage nodes therefore depends on the types and timespan of the queries (which affects where the segments are located and if they need to be recovered from bucket storage).
To monitor the performance use the Insights Segments & Datasources Dashboard. Also, see the Monitoring Capacity Metrics documentation.