Array Syntax

The documentation explains array syntax in LogScale, covering both flat and nested arrays, including how arrays are structured, accessed, and manipulated using JSON-like notation with square brackets and dots. It details structured array syntax for working with arrays of objects, array notation rules for field naming, and provides comprehensive examples of array operations using functions like array:contains(), array:eval(), and objectArray:eval().

Falcon LogScale operates on two types of arrays:

  • Flat arrays

  • Nested arrays

Nested arrays are fields that consist of an array where each element is another array or object. There are two types of nested arrays: arrays of arrays (such as foo[0][1]) and arrays of objects (such as users[0].username).

While there is no distinct array type within LogScale, the system is capable of operating on array-like objects using syntax similar to that used for JSON arrays and objects. In this context, an array is defined as an ordered collection of values: each value is called an element, and each element has a numeric position in the array, known as its index.

Arrays can be parsed from incoming events using these two functions:

  1. splitString()

  2. parseJson()

The CrowdStrike Query Language (CQL) includes syntax that applies to array functions, called array syntax. Array syntax can be used on flat or nested arrays. The function objectArray:eval() operates specifically on arrays of objects (a type of nested array), and provides structured array syntax for accessing object fields within each array element. Note that it can only read the structured array fields, it cannot write to them.

Array functions operate on fields that follow this specific naming convention syntax.

When referring to an entire array, specify the name of the field and append square brackets []. For example, to interpret usernames as an array: 

usernames[]

When composing an array of files, the array syntax is similar to the one used by JSON, where [ and ] are used for indexing within arrays and . for selecting members in objects. The following example is an array with two elements, with the first element username[0] containing the value jsmith:

json
["jsmith","tmcdonald"]

In the next example, the object username can be extracted using user.username:

json
{username: "jsmith", name: "Jon Smith"}

The following table compares the JSON and the parsed LogScale form of the same array and object structures. The final column shows the syntax to use when referring to the whole array.

JSON Parsed LogScale event Array Name
["jsmith","tmcdonald"]

username[0]: "jsmith"

username[1]: "tmcdonald"

username[] 
[
  {username: "jsmith", name: "Jon Smith"},
  {username: "jdoe", name: "Jane Doe"}
]

users[0].user.username: "jsmith"

users[0].user.name: "Jon Smith"

users[1].user.username: "jdoe"

users[1].user.name: "Jane Doe"

users[]
[ ["a", "b"], ["c", "d"]]

foo[0][0]: "a"

foo[0][1]: "b"

foo[1][0]: "c"

foo[1][1]: "d"

foo[], foo[0][], and foo[1][]

Structured Array Syntax

Structured array syntax is a special notation for accessing fields within arrays of objects. It provides dot notation (such as in.key) for accessing object fields and bracket notation (such as in[0]) for indexing that can be combined to access nested data structures. This syntax is used primarily with the objectArray:eval() function.

Let's start with an example usage:

logscale Syntax
objectArray:eval("in[]", asArray="out[]", function={out := format("%s:%s", field=[in.key, in.value])})
input:
       in[0].key = x
       in[0].value = y
       in[1].key = a
       in[1].value = b
output:
       out[0] = x:y
       out[1] = a:b

objectArray:eval("in[]", asArray="out[]", function=function) iterates over the array from start to end (or to the first "hole" in the array).

For each array entry, the given function is applied.

For each entry, at index i, a field out[i] is created if the function writes a value to the output array name field ("out" in this example).

Within the function, a special pattern to access sub-selections of array entries is supported.

This pattern is:

in (the input array) followed by:

  • .<subselection>

  • or [<number>]

  • or any combination of the above

Examples:

  • in.key

  • in.others[0].foo

  • in[0][1]

This pattern is what allows the function to operate on arrays of objects (such as in.key), and arrays of arrays (such as in[0][1]), and other arrays of structured data.

Semantically, given the input array "in", an array index i, and an access in.<subselection> this will be translated to the field name in[i].<subselection>. Similarly, in[2] is translated to in[i][2].

The character sequence could contain another dot. For example in.foo.bar becomes in[i].foo.bar.

At present, it is only possible to read structured array fields. Writing to them is unsupported, but is a possible future extension. For that reason, at present we disallow modifying the existing array.

The var parameter can be used to give a different name to the input array variable inside the function argument. This is particularly useful whenever the input array name is very long. Example:

objectArray:eval("someVeryLongName[]", asArray="out[]", var=x, function={out := format("%s:%s", field=[x.key, x.value])})

Syntax Array Notation

Fields with names consisting of a valid array-prefix followed by an array entry are treated as entries of an array. The indices must be continuously increasing from 0 and upwards.

The entire array is referenced within array function using the array-prefix followed by [] that indicates the entire range of entries, for example username[].

Simplified, the valid array-prefix is a sequence of members and array entries. More formally, a valid array-prefix is either:

  • an identifier — that is, a sequence of alphanumerical characters that starts with a letter, an underscore (_), @, or #.

    If the member name is not an identifier, for example because it starts with a number or contains other special characters, the name needs to be quoted using ` like this: user.`e-mail`[].

  • a valid array-prefix followed by a member operator and an identifier — the member operator is a .

  • or a valid array-prefix followed by an array entry — the array entry is an index surrounded by square brackets.

Examples of field names representing entries in an array, and the reference to that array:

Field names following the array syntax How to reference in array functions
a[0], a[1], a[2] a[]
a.b[0], a.b[1], a.b[2] a.b[]
a[0][0], a[0][1] a[0][]
user.e-mail[0], user.e-mail[1] user.`e-mail`[]

Warning

If the array is a structured array, for example, a[][0] or a[].b, use the objectArray: functions.

For arrays, use the array: functions.