Array Operations
Array Concatenation
To concatenate an array, use
If you have an array want and to concatenate the contents, irrespective
of how many elements there might be in the array, you can use a
combination of objectArray:eval() in combination
Array Length for Nested Arrays
To get the length of a flat array, use
array:length():
array:length(myArray[])However this will not work for nested arrays, so if you have the following nested array:
myArray.nested[0].count := 1
| myArray.nested[1].count := 2
| array:length("myArray.nested[]")
The count will return 0. To get an accurate count of the number of items
in the array, you must compose a new simpler array and count that using
objectArray:eval():
myArray.nested[0].count := 1
| myArray.nested[1].count := 2
| objectArray:eval(array="myArray.nested[]", asArray="arraycount[]", function={arraycount:= true})
| array:length("arraycount[]")
| array:drop("arraycount[]")This method:
Creates a new array, arraycount with an element for each element of the nested array.
Counts the length of the new array
Depetes the temporary array so that it does not appear in later events.
This returns the expected result of 2 elements.
Access an Array Element by the Index of Another Field
To get the element of an array by using the value of another field as an
the index value, you can combine the getField()
action, which gets field values based on the string name by creating the
string for the field.
For example, consider the array arrayField[]:
["a","b","c"]Using the following query:
indexField := 2
| output := getField(format("arrayField[%s]", field=[indexField]))
Sets output to c.
The way this works is to construct the fieldname using
format, so that:
format("arrayField[%s]", field=[indexField])
Returns the string arrayField[2], then
getField() accesses the field name just as if it
had been typed in the query.
Creating an Array in a Query
Assignment of an array to a field is not supported, so the query fragment:
myArray = [1,2,3]
Does not work. Instead, use array:append() to add
new elements to an array defined in the function:
array:append(array="myArray[]",values=[1,2,3])Getting Length of an Object Array
Getting the length of complex arrays, or object arrays where there
multiple levels of content, array:length() may be
not calculate the right length of the content. The solution is to use
objectArray:eval() to iterate over the object array
and create a new array based on the individual elements of the
object-based array structure:
objectArray:eval(array="a.b[]", asArray="flat[]", function={flat:=""})
| array:length("flat[]", as=_codeAnalysesLength)Note that this creates a temporary field that you may want to remove so that it does not appear in later stages of processing and reduce the memory overhead. To do this, you could add:
array:drop("flat[]")To your query.
Difference Between Arrays
To determine the difference between two different arrays, you will need to iterate over the array and compare each element of array1[] with all of the elements of array2[]. As such, this can be quite an expensive operation if the arrays are quite large.
Sample code for this is shown the example below:
array:append(array="array1[]", values=["1","2","3","5"])
| array:append(array="array2[]", values=["1","4","6","7"])
| array:filter(
array="array1[]",
var="elem1",
function={
!array:exists(
array="array2[]",
var="elem2",
condition={test(elem1 == elem2)}
)
},
asArray="diff[]"
)
| concatArray(diff, separator=", ", as="difference")This works by:
Iterate over array1.
Compare each element of array1 by using
array:exists()to determine the element exists in array2.As the new element to diff[] if the value does not exist.
Creates a field difference as a comma separated list of the values.
Flattening Arrays
Sometimes you may have an array that has a variable number of elements, but you want to be able to query, group or summarize by that content. For example:
| {"groups": ["users","admins","printers"],"user":"asmith"} |
|---|
| {"groups":["users","printers"],"user":"bjones"} |
| {"groups":["printers"],"user":"cryan"} |
Generates an array for each event with a variable number of elements,
and inconsistent assignment; as in, orinters appears
as element 0, 1 and 2 across the three events:
| groups[0] | groups[1] | groups[2] | user |
|---|---|---|---|
| users | admins | printers | asmith |
| users | printers | <no value> | bjones |
| printers | <no value> | <no value> | cryan |
Performing a groupBy() on
groups[] (an array) is not supported, and selecting
a single element will only group on that element (there is no value for
element 2 of the array in the third event).
The solution to this and many other situations where you have a variable
number of elements in an array and still want to report or summarize on
the content is to use the split(). This duplicates
event, but replaces the array with each element. So from our example
above, running:
split(groups)Into the resultset:
| user | groups |
|---|---|
| asmith | printers |
| asmith | admins |
| asmith | users |
| bjones | printers |
| bjones | users |
| cryan | printers |
Note how we only have two fields, no arrays, and six events (three from the first array, two from the second, one from the third).
Now we can use groupBy() on the non-array field
groups:
split(groups)
| groupBy(groups);Produces:
| groups | _count |
|---|---|
| admins | 1 |
| printers | 3 |
| users | 2 |
Filtering on Array
To filter on a value in a array when you do not know which element of
the array might contain the value, use the
concatArray() function to turn the array into a
string. Now you can filter on that value. For example, using the same
data set as above, to extract all the users that are a member of the
users group:
groupfilter := concatArray(groups)
| /users/This creates a new field, groupfilter which we can then filter on:
"bjones","usersprinters"
"asmith","usersadminsprinters"