Search using a CSV or JSON file and enrich entries.
To use this function, you can use the UI to upload a file using the Files menu, or can upload a CSV or JSON file using the lookup api.
You can use it to do something like field IN xxx, where xxx is really all
the values in a column=Name
in the CSV
file=File.CSV
you specify.
The default behavior of this function — when
strict
is set to true — works like an
INNER JOIN
. When strict
is set to
false
, it works like the deprecated
lookup()
function (i.e., it just enriches events that
match), but let all events pass through even if they don't match.
If using glob=true, the underlying CSV is limited to 20000 rows/lines. It
is configurable using the config parameter
MAX_STATE_LIMIT
. For exact matching glob=false the file
is limited to 1000000 rows/lines by default and can be configured using
the parameter EXACT_MATCH_LIMIT
.
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
column | string | false | Specifies which column in the file to use for the match. Defaults to the value of the field parameter. | |
field | string | true | Specifies which field in the event (log line) that must match the given column value. | |
file | string | true | Specifies the source file. | |
glob | boolean | false | false | If true, the key column in the underlying file is interpreted as a globbing pattern with * . |
ignoreCase | boolean | false | false | If true, ignore case when matching against the CSV data. |
include | string | false | Specifies columns to include. If no argument given, include all columns from the corresponding row in the output event. | |
mode | string | false | string | What function to use when matching against keys. |
strict | boolean | false | true | If true (default) only field events that match a key in the file; if false let all events through (works like the deprecated lookup() ). |
Examples
Matches events for which the id
field matches the
value of the column in the table "users.csv". Does not add any columns.
match(file="users.csv", column=userid, field=id, include=[])
Matches events for which the id
field is matched
case-insensitive by the glob-pattern in the column
userid
in the table users.csv
, and
add all other columns of the first matching row to those events.
id =~ match(file="users.csv", column=userid, mode=glob, ignoreCase=true)
Let all events pass through, but events for which the
id
field matches the value of the
userid
column in the table
users.csv
will be enriched with all other columns of
the matching row.
id =~ match(file="users.csv", column=userid, strict=false)
Matches events for which the ip
field matches the
CIDR subnet of the cidr-block
column in the table
cidr-file.csv
. Only adds the columns
info
and type
from the first
matching row.
match(file="cidrfile.csv", column="cidr-block", field=ip, mode=cidr, include=["info","type"])