Effects of Limits
Limits within LogScale manage query execution by limiting and/or balancing the process of execution across the cluster. They limit event output and help manage cluster resources, preventing one or more queries from using too many resources and causing cluster failure. This contributes to ensuring efficient execution of all queries.
To assist with this at a basic level and to avoid a simple query returning a high number of events from a basic filter, LogScale automatically applies the Implied Limit.
Other limits include the following:
Function Limits affect the operation of a single function within the overall query.
Variables Governing Limits affect the default or maximum limits of functions, or the execution of the overall query.
Different limits work in tandem with one another- a function-specific limit only affects that function within the query, but the limit will then also be controlled by dynamic limits. For example, consider the following limit:
#repo=A session_id=*
| !join(query={#repo=B session_id=*}, field=session_id, key=session_id)
| groupBy([user,session_id],limit=max)
The dynamic configuration parameter JoinRowLimit restricts
the number of matching rows for the join() function,
and the dynamic configuration parameter GroupMaxLimit
restricts the number of matching rows for for the
groupBy() function. The whole query will be limited
by the dynamic configuration parameters
QueryResultRowCountLimit and StateRowLimit.
| Item | Affects | Example |
|---|---|---|
| Function-controlled Limits | Limits controlled through a function parameter |
tail(limit),sort(limit)
|
| Variable Limits on Function Parameters | Limit the maximum values that can be set by a function parameter |
GroupDefaultLimit, JoinRowLimit
|
| Variable Limits on Functions | Limit the maximum operations or events for a function |
GroupDefaultLimit, JoinRowLimit
|
| Limits for Query Overall | Limit the maximum operations or events for a function |
AggregatorOutputRowLimit,
QueryResultRowCountLimit
|
| Limits for LogScale Environment | Limit the resource usage overall |
QueryCoordinatorMemoryLimit
|
For more information on function limits, see Function Limits. For more information on variables that control limits, see Variables Governing Limits.