Deduplicating fields of information where there are multiple occurrences of a value in a single field, maybe separated by a single character can be achieved in a variety of ways. This solution uses array:union() and split create a unique array and then split the content out to a unique list.

For example, when examining the humio and looking for the browsers or user agents that have used your instance, the UserAgent data will contain the browser and toolkits used to support them, for example:

The actual names are the Name/Version pairs showing compatibility with different browser standards. Resolving this into a simplified list requires splitting up the list, simplifying (to remove duplicates), filtering, and then summarizing the final list.

The resulting output from the query is a list of events with each event containing a matching _index and browser. This can be useful if you want to perform further processing on a list of events rather than an array of values.

It is possible to filter an array on a given condition using the array filter function array:filter(). The array:filter() creates a new array with elements matching the specified conditions and does not change the original array. The new array will retain the original order.

Example incoming data might look like this:

mailto[0]=foo@example.com
mailto[1]=bar@example.com
mailto[2]=baz@example.com

The query is used to filter values from the input array using the function provided in the array and return a new array with the results meeting the specified condition.

Sample output from the incoming example data:

out[0]=bar@example.com
out[1]=baz@example.com