Parsers Throttling

Query

logscale

#kind=logs class=/ParserLimitingJob/ "Top element for parser id"
pct:=100*costSum/threshold
timeChart(function=max(pct), minSpan=10s)

Introduction

Throttling is used to maintain the optimal performance and reliability of the system, as throttling limits the number of API calls or operations within a time window to prevent the overuse of resources.

In this example, the timeChart() function is used to show how close (in percentage) the system has been to start throttling any parser.

Step-by-Step

Starting with the source repository events.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] 1[/Filter/] 2[/Filter/] result{{Result Set}} repo --> 0 0 --> 1 1 --> 2 2 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
#kind=logs class=/ParserLimitingJob/ "Top element for parser id"
```
Filters on all logs in humio that are tagged with kind equal to logs and then returns the events where the class field has values containing /ParserLimitingJob/, and where the logline contains the string Top element for parser id.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] 1[/Filter/] 2[/Filter/] result{{Result Set}} repo --> 0 0 --> 1 1 --> 2 2 --> result style 1 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
pct:=100*costSum/threshold
```
Calculates the percentage of the values in the field costSum divided with values in the field threshold and returns the results in a new field named pct.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] 1[/Filter/] 2[/Filter/] result{{Result Set}} repo --> 0 0 --> 1 1 --> 2 2 --> result style 2 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
timeChart(function=max(pct), minSpan=10s)
```
Shows the calculated sum of the max values in the field pct in percentage in spans of 10 sec in a timechart.
Event Result set.