Effects of Limits

Limits within LogScale manage query execution by limiting and/or balancing the process of execution across the cluster. They limit event output and help manage cluster resources, preventing one or more queries from using too many resources and causing cluster failure. This contributes to ensuring efficient execution of all queries.

To assist with this at a basic level and to avoid a simple query returning a high number of events from a basic filter, LogScale automatically applies the Implied Limit.

Other limits include the following:

Different limits work in tandem with one another- a function-specific limit only affects that function within the query, but the limit will then also be controlled by dynamic limits. For example, consider the following limit:

logscale
#repo=A session_id=*
| !join(query={#repo=B session_id=*}, field=session_id, key=session_id)
| groupBy([user,session_id],limit=max)

The dynamic configuration parameter JoinRowLimit restricts the number of matching rows for the join() function, and the dynamic configuration parameter GroupMaxLimit restricts the number of matching rows for for the groupBy() function. The whole query will be limited by the dynamic configuration parameters QueryResultRowCountLimit and StateRowLimit.

Item Affects Example
Function-controlled Limits Limits controlled through a function parameter tail(limit),sort(limit)
Variable Limits on Function Parameters Limit the maximum values that can be set by a function parameter GroupDefaultLimit, JoinRowLimit
Variable Limits on Functions Limit the maximum operations or events for a function GroupDefaultLimit, JoinRowLimit
Limits for Query Overall Limit the maximum operations or events for a function AggregatorOutputRowLimit, QueryResultRowCountLimit
Limits for LogScale Environment Limit the resource usage overall QueryCoordinatorMemoryLimit

For more information on function limits, see Function Limits. For more information on variables that control limits, see Variables Governing Limits.