Matches a value in the CSV or through a limited form of JSON file, uploaded using Lookup Files.

The default behavior of this function — when strict is set to true — works like an INNER JOIN. When strict is set to false the function enriches events.

When using glob=true, the underlying CSV is limited to 20,000 rows/lines. For exact matching glob=false the file is limited to 1,000,000 rows/lines.

The maximum value for glob matches is configurable using MAX_STATE_LIMIT. The maximum value for exact matches is configurable using EXACT_MATCH_LIMIT.

Function Traits: Negatable, Transformation

ParameterTypeRequiredDefaultDescription
columnstringoptionalfield parameter Specifies which column in the file to use for the match.
fieldstringrequired  Specifies which field in the event (log line) that must match the given column value.
file[a]stringrequired  Specifies the source file.
globbooleanoptionalfalse If true, the key column in the underlying file is interpreted as a glob pattern if a * is included in the key value, e.g. a CSV key value of *thisMatch* would match the field value of 123thisMatch456.
ignoreCasebooleanoptionalfalse If true, ignore case when matching against the CSV data.
includestring or arrayoptional  Specifies columns to include. If no argument given, include all columns from the corresponding row in the output event.
modestringoptionalstring The function to use when matching against keys.
  Valid ValuescidrThe key is interpreted as a CIDR subnet and the event is matched if the field contains an IP within the subnet. If multiple subnets match, the most specific one is selected or an arbitrary one if there are multiple equally specific subnets.
   globThe key is interpreted as a globbing pattern with * and matched accordingly.
   stringThe matching is done using exact string matching.
strictbooleanoptionaltrue If true (default) only field events that match a key in the file; if false let all events through (works like the deprecated lookup()).

[a] The argument name file can be omitted.

The parameter name for file can be omitted; the following forms are equivalent:

logscale
match("value")

and:

logscale
match(file="value")

match() File Formats

For Comma Separated Values (CSV) files, whitespace gets included in the keys and values. To include the separator "," in a value, quote using the " character. The following file is a valid CSV file:

csv
userid,name
1,chr
2,krab
"4","p,m"
7,mgr

The first line is intepreted as a the column titles. When querying, the column in the field should be used to identify which column to match against.

For JSON files, two formats are supported:

  • Object-based, where the lookup field does not have an explicit name

  • Array-based, where the information is an array of objects

In the Object-based variant, the lookup values are declared as an object with a key and embedded fields, the key field does not have a name.

json
{
  "1": { "name": "chr" },
  "2": { "name": "krab" },
  "4": { "name": "pmm" },
  "7": { "name": "mgr" }
}

When matching against a file in this case, the name of the field in the JSON object does not need to be used; the key for each value is used instead. For example:

logscale
groupBy(@timezone)
| count(@timezone)
| match(file="short.json",field=_count)

In the above, the value of _count will be matched, outputting the match value: 

_count    name
       2    krab

In the Array-based variant, the lookup values are declared as an array of objects, you select which field is the key using the field parameter in match().

json
[
  { "userid": "1", "name": "chr" },
  { "userid": "2", "name": "krab" },
  { "userid": "4", "name": "pmm" },
  { "userid": "7", "name": "mgr" }
]

When using this version, the name of the column to be matched must be specified using the column argument to match():

logscale
groupBy(@timezone)
| count(@timezone)
| match(file="long.json",field=_count,column="userid")

This behavior also means that any field in the JSON file can be used as the match value. For example:

logscale
...
| match(file="long.json",field=codename,column="name")

This can be useful if you have a JSON file that contains multiple possible lookup values for given records.

Important

The match() does not report an error if the file format cannot be parsed.

match() Examples

Matches events for which the id field matches the value of the column in the table "users.csv". Does not add any columns.

logscale
match(file="users.csv", column=userid, field=id, include=[])

Matches events for which the id field is matched case-insensitive by the glob-pattern in the column userid in the table users.csv, and add all other columns of the first matching row to those events.

logscale
id =~ match(file="users.csv", column=userid, mode=glob, ignoreCase=true)

Let all events pass through, but events for which the id field matches the value of the userid column in the table users.csv will be enriched with all other columns of the matching row.

logscale
id =~ match(file="users.csv", column=userid, strict=false)

Matches events for which the ip field matches the CIDR subnet of the cidr-block column in the table cidr-file.csv. Only adds the columns info and type from the first matching row.

logscale
match(file="cidr-file.csv", column="cidr-block", field=ip, mode=cidr, include=["info","type"])