Array Operations

Array Concatenation

To concatenate an array, use

If you have an array want and to concatenate the contents, irrespective of how many elements there might be in the array, you can use a combination of objectArray:eval() in combination

Array Length for Nested Arrays

To get the length of a flat array, use array:length():

logscale
array:length(myArray[])

However this will not work for nested arrays, so if you have the following nested array:

Invalid Example for Demonstration - DO NOT USE
logscale
myArray.nested[0].count := 1
| myArray.nested[1].count := 2
| array:length("myArray.nested[]")

The count will return 0. To get an accurate count of the number of items in the array, you must compose a new simpler array and count that using objectArray:eval():

logscale
myArray.nested[0].count := 1
| myArray.nested[1].count := 2
| objectArray:eval(array="myArray.nested[]", asArray="arraycount[]", function={arraycount:= true})
| array:length("arraycount[]")
| array:drop("arraycount[]")

This method:

  • Creates a new array, arraycount with an element for each element of the nested array.

  • Counts the length of the new array

  • Depetes the temporary array so that it does not appear in later events.

This returns the expected result of 2 elements.

Access an Array Element by the Index of Another Field

To get the element of an array by using the value of another field as an the index value, you can combine the getField() action, which gets field values based on the string name by creating the string for the field.

For example, consider the array arrayField[]:

logscale
["a","b","c"]

Using the following query:

logscale
indexField := 2
| output := getField(format("arrayField[%s]", field=[indexField]))

Sets output to c.

The way this works is to construct the fieldname using format, so that:

logscale
format("arrayField[%s]", field=[indexField])

Returns the string arrayField[2], then getField() accesses the field name just as if it had been typed in the query.

Creating an Array in a Query

Assignment of an array to a field is not supported, so the query fragment:

Invalid Example for Demonstration - DO NOT USE
logscale
myArray = [1,2,3]

Does not work. Instead, use array:append() to add new elements to an array defined in the function:

logscale
array:append(array="myArray[]",values=[1,2,3])

Getting Length of an Object Array

Getting the length of complex arrays, or object arrays where there multiple levels of content, array:length() may be not calculate the right length of the content. The solution is to use objectArray:eval() to iterate over the object array and create a new array based on the individual elements of the object-based array structure:

logscale
objectArray:eval(array="a.b[]", asArray="flat[]", function={flat:=""})
| array:length("flat[]", as=_codeAnalysesLength)

Note that this creates a temporary field that you may want to remove so that it does not appear in later stages of processing and reduce the memory overhead. To do this, you could add:

logscale
array:drop("flat[]")

To your query.

Difference Between Arrays

To determine the difference between two different arrays, you will need to iterate over the array and compare each element of array1[] with all of the elements of array2[]. As such, this can be quite an expensive operation if the arrays are quite large.

Sample code for this is shown the example below:

logscale
array:append(array="array1[]", values=["1","2","3","5"])
| array:append(array="array2[]", values=["1","4","6","7"])
| array:filter(
    array="array1[]",
    var="elem1",
    function={
      !array:exists(
        array="array2[]",
        var="elem2",
        condition={test(elem1 == elem2)}
      )
    },
    asArray="diff[]"
)
| concatArray(diff, separator=", ", as="difference")

This works by:

  • Iterate over array1.

  • Compare each element of array1 by using array:exists() to determine the element exists in array2.

  • As the new element to diff[] if the value does not exist.

  • Creates a field difference as a comma separated list of the values.

Flattening Arrays

Sometimes you may have an array that has a variable number of elements, but you want to be able to query, group or summarize by that content. For example:

{"groups": ["users","admins","printers"],"user":"asmith"}
{"groups":["users","printers"],"user":"bjones"}
{"groups":["printers"],"user":"cryan"}

Generates an array for each event with a variable number of elements, and inconsistent assignment; as in, orinters appears as element 0, 1 and 2 across the three events:

groups[0]groups[1]groups[2]user
usersadminsprintersasmith
usersprinters<no value>bjones
printers<no value><no value>cryan

Performing a groupBy() on groups[] (an array) is not supported, and selecting a single element will only group on that element (there is no value for element 2 of the array in the third event).

The solution to this and many other situations where you have a variable number of elements in an array and still want to report or summarize on the content is to use the split(). This duplicates event, but replaces the array with each element. So from our example above, running:

logscale
split(groups)

Into the resultset:

usergroups
asmithprinters
asmithadmins
asmithusers
bjonesprinters
bjonesusers
cryanprinters

Note how we only have two fields, no arrays, and six events (three from the first array, two from the second, one from the third).

Now we can use groupBy() on the non-array field groups:

logscale
split(groups)
| groupBy(groups);

Produces:

groups_count
admins1
printers3
users2

Filtering on Array

To filter on a value in a array when you do not know which element of the array might contain the value, use the concatArray() function to turn the array into a string. Now you can filter on that value. For example, using the same data set as above, to extract all the users that are a member of the users group:

logscale
groupfilter := concatArray(groups)
| /users/

This creates a new field, groupfilter which we can then filter on:

logscale
"bjones","usersprinters"
"asmith","usersadminsprinters"