Deduplicate Values in Array
Remove duplicate values in an array using array:dedup()
Query
logscale
parseJson()
| array:dedup("emails[]")
Introduction
The array:dedup()
function can be used to
remove duplicate values in an array.
In this example, the function removes duplicates in the array emails[].
Example incoming data might look like this:
Raw Events
{"emails": ["john@mail.com", "admin@mail.com", "jane@mail.com", "admin@mail.com"]} |
Step-by-Step
Starting with the source repository events.
- flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] 1{{Aggregate}} result{{Result Set}} repo --> 0 0 --> 1 1 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;logscale
parseJson()
Parses the incoming data to identify JSON values and converts them into a usable field.
- flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] 1{{Aggregate}} result{{Result Set}} repo --> 0 0 --> 1 1 --> result style 1 fill:#ff0000,stroke-width:4px,stroke:#000;logscale
| array:dedup("emails[]")
Removes duplicate values in the array emails[].
Event Result set.
Summary and Results
The query is used to remove duplicate values in an array.
Sample output from the incoming example data:
emails[0] | emails[1] | emails[2] |
---|---|---|
john@mail.com | admin@mail.com | jane@mail.com |