Parse events encoded with key/value pairs into individual fields and values. This function can run an extra key/value parser on events.
Parameter | Type | Required | Default Value | Description | |
---|---|---|---|---|---|
as | string | optional[a] | Prefix for all resolved field keys. | ||
excludeEmpty | array of strings | optional[a] | false | If the value of a key is empty, exclude the field. | |
field [b] | Array of strings | optional[a] | @rawstring | Fields that should be key-value parsed. | |
onDuplicate | string | optional[a] | keepLast | Set the value for duplicate keys that exist in the event. | |
Valid Values | |||||
keepFirst | Keep the first duplicate value | ||||
keepLast | Keep the last duplicate value | ||||
override | boolean | optional[a] | false | Override existing values for keys that already exist in the event. | |
separator | string | optional[a] | = | The token that separates the key from the value — a single char only. | |
separatorPadding | string | optional[a] | unknown | Help the function recognize unquoted empty values and parse them by specifying whether there is a whitespace around the key-value separator (typically = ). For a list of interpretations, see kvParse() separatorPadding argument handling. | |
Valid Values | |||||
no | Assumes the source does not have a whitespace around the key-value
separator, as in a=1, b=2 | ||||
unknown | Whether the source has a padding (whitespace) around values is not recognized. | ||||
yes | Indicates the source has a whitespace around the key-value separator, as
in a = 1, b = 2 | ||||
[a] Optional parameters use their default value unless explicitly set. |
Hide omitted argument names for this function
Omitted Argument NamesThe argument name for
field
can be omitted; the following forms of this function are equivalent:logscalekvParse("field")
and:
logscalekvParse(field="field")
These examples show basic structure only.
It is used to parse key/values of the form:
key=value
key="value"
key='value'
key = value
Both key and value can be either quoted using
"
or '
, or
unquoted. If using quotes, the quotes must be terminated.
For a log line like this:
2017-02-22T13:14:01.917+0000 [main thread] INFO UserService - creating new user id=123, name='john doe' email=john@doe
The key/value parser extracts the fields:
id=123
name=john doe
email=john@doe
Use the parameter field
to
specify which fields should be key/value parsed. Specifying
@rawstring to key/value parse the rawstring.
kvParse()
Examples
Key/value parse the log line:
creating new user id=123, name='john doe' email=john@doe.
This will add the fields id=123, name='john doe' and email=john@doe to the event:
kvParse()
Key/value parse the log line:
creating new user id=123, name='john doe' email=john@doe loglevel=ERROR.
Assuming the event already has a loglevel field,
replacing the value of that field with ERROR
requires
parameter override=true
:
kvParse(override=true)
Key/value parse a nested field. In this example we will use JSON input:
{
"service": "paymentService",
"type": "payment",
"metadata": "host=server5,transactionID=123,processingTime=100"
}
and parse out the key/values in the metadata field:
parseJson()
| kvParse(metadata)
Key/value parse the log line and export fields with a prefix:
creating new user id=123, name='john doe' email=john@doe.
This will add the fields user.id=123, user.name='john doe' and user.email=john@doe to the event:
kvParse(as="user")
Key/value parse the log line:
firstname = John middlename = lastname = Doe
This will add the fields firstname=John,
middleName= (empty value) and
lastname=Doe to the event with a whitespace around
the key-value separator (=
):
kvParse(separatorPadding="yes")
When parsing a key/value line, the impact of spacing between the key, value and equals sign can lead to interpretation differences. The
separatorPadding
parameter controls this by defining how different patterns are interpreted with and without spacing, as follows:Raw Data separatorPadding Field a Value Field b Value Notes a = b = c unknown
b c dropped a = b = c yes
(Empty) c a = b = c no
(Empty) (Empty) c dropped a=b=c unknown
b=c a=b=c yes
b=c a=b=c no
b=c a = b=c unknown
b=c a = b=c yes
b=c a = b=c no
(Empty) c c dropped a=b = c unknown
b a=b = c yes
(Empty) (Empty) c dropped a=b = c no
b
Keep the first value for duplicated keys.
ininame='john doe' name='jane doe'
By default
kvParse()
will keep the last seen value. To keep the first value instead, set as follows:logscalekvParse(onDuplicate=keepFirst)
name john doe Keep the first value for duplicated keys, with a preset field: if name is set to
alice
beforehand, thenonDuplicate=keepFirst
parameter has no effect and name will keep such preset value. If you want to ignore the preset value, use theoverride
parameter.ininame='john doe' name='jane doe'
logscalekvParse(onDuplicate=keepFirst, override=true)
name john doe Keep the last value for duplicated keys, with a preset field: if name is set to
alice
beforehand, then theonDuplicate=Last
parameter has no effect and name will keep such preset value. If you want to ignore the preset value, use theoverride
parameter.ininame='john doe' name='jane doe'
logscalekvParse(onDuplicate=keepLast, override=true)
name jane doe