Deduplicate Content by Field
Deduplicating content based on a specific field
Query
logscale
groupBy(field, function=tail(1))
Introduction
If you want to deduplicate events by a given field, for example to
identify a unique list of events for further processing, you can
use an aggregate function. In this example, the
groupBy()
function is used with
tail()
to use the last value in a sequence of
events.
Step-by-Step
Starting with the source repository events.
- flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0{{Aggregate}} result{{Result Set}} repo --> 0 0 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;logscale
groupBy(field, function=tail(1))
Groups all events in a specific field, and reduces the results using
tail()
to take only the last value. Event Result set.
Summary and Results
The query is used to deduplicate events by a given field. This is useful if you want to create a unique list of events for further processing.