Parses data as JSON. Specify
field=@rawstring
to parse the rawstring
into JSON. It is possible to prefix the names of the extracted fields
using the prefix parameter. It is also possible to exclude some of the
extracted fields using the exclude parameter, specify
exclude=a.b.c
to exclude
c
and all of its descendants or
exclude="a.b[*].c"
to exclude all
c
inside the array
b
. If you need to keep certain descendants
of an otherwise excluded path you can use the include parameter.
Parameter | Type | Required | Default Value | Description | |
---|---|---|---|---|---|
exclude | Array of strings | optional[a] | [] | Fields that should be excluded from the result, supports dot-pathing and array wildcards. If used with prefix the exclude fields should be prefixed as well. | |
excludeEmpty | boolean | optional[a] | false | Whether to exclude if the field is empty. | |
Valid Values | |||||
false | Don't exclude the field, even if the value is empty | ||||
true | Exclude the field if the value is empty | ||||
field [b] | string | required | @rawstring | Fields that should be parsed as JSON. | |
handleNull | string | optional[a] | keep | How null values are handled. | |
Valid Values | |||||
discard | Discard the null value and field null value with an empty string
"" | ||||
empty | Replaces a null value with an empty string "" | ||||
keep | Converts the value to the "null" string | ||||
include | Array of strings | optional[a] | [] | Fields that should be included even if they had been previously excluded by use of exclude , supports dot-pathing and array wildcards. If used with prefix the include fields should be prefixed as well. | |
prefix | string | optional[a] | blank | Prefix the name of the extracted JSON fields with the value of this parameter. | |
removePrefixes | Array of strings | optional[a] | [] | Prefixes that should be removed from the names of the extracted JSON fields, supports dot-pathing. If multiple prefixes are supplied, the longest matching prefix will be used. | |
[a] Optional parameters use their default value unless explicitly set. |
Hide omitted argument names for this function
Omitted Argument NamesThe argument name for
field
can be omitted; the following forms of this function are equivalent:logscaleparseJson("field")
and:
logscaleparseJson(field="field")
These examples show basic structure only.
When parsing JSON, the following apply:
When
excludeEmpty=true
is used, the key/value pairs will be discarded completely whenever the original json containsfoo: ""
When the JSON contains
foo: null
, usinghandleNull=discard
then the entire key/value pair is discarded, regardless of the setting forexcludeEmpty
parseJson()
Examples
If the whole event sent to LogScale is JSON like:
{"service": "userService", "timestamp": "2017-12-18T20:39:35Z", "msg": "user with id=47 logged in"}
parseJson()
| parseTimestamp(field=timestamp)
If a field in the incoming event contains JSON like:
2017-12-18T20:39:35Z user id=47 logged in details="{"name": "Peter", "email": "peter@test.com", "id":47}"
In the example below the details field is extracted using the kvparse function and then parseJson is used to parse the JSON inside the details field.
/(?<timestamp>\S+)/
| parseTimestamp(field=timestamp)
| kvParse()
| parseJson(field=details)
It is possible to prefix names of the extracted JSON fields. This can be useful for avoiding collisions with existing fields with the same name. For example the input line:
added new user details="{"email": "foo@test.com", "name": "Peter"}"
Could be parsed into these fields:
<user.email=foo@test.com>
, user.name=Peter.
kvParse()
| parseJson(field=details, prefix="user.")
It is possible to remove prefixes as well. For example the input line:
details="{"a": { "b": { "c": { "d": "e", "f": "g"}, "h": "i" }, "j": "k" } }"
If the JSON object is expanded:
{
"a" : {
"b" : {
"c" : {
"d" : "e",
"f" : "g"
},
"h" : "i"
},
"j" : "k"
}
}
Would be parsed into these fields with the following function if the
a
prefix is removed: b.c.d=e
,
b.c.f=g
, b.h=i
, j=k
.
kvParse()
| parseJson(field=details, removePrefixes=a.)
It is possible to exclude extracted fields. This can be useful for removing sensitive data or, for example, large arrays. For example, the input line:
details="{"a": { "b": { "c": { "d": "e", "f": "g"}, "h": "i" }, "j": "k" } }"
If the raw JSON is formatted:
{
"a" : {
"b" : {
"c" : {
"d" : "e",
"f" : "g"
},
"h" : "i"
},
"j" : "k"
}
}
The JSON string would be parsed into these fields: a.b.h=i
,
a.j=k
but not, for example, a.b.c.d=e
kvParse()
| parseJson(field=details, exclude=a.b.c)
It is also possible to exclude extracted fields within arrays. For example the input line:
details="{"a": { "b": [{ "c": { "d": 1 }, "e": "f" }, { "c": { "d": 2 }, "e": "h" }] } }"
As a formatted JSON string:
{
"a" : {
"b" : [
{
"c" : {
"d" : 1
},
"e" : "f"
},
{
"c" : {
"d" : 2
},
"e" : "h"
}
]
}
}
Would be parsed into these fields: a.b[0].e=f
,
a.b[1].e=h
but not, for example, a.b[0].c.d=1
.
kvParse()
| parseJson(field=details, exclude="a.b[*].c")
It is possible to include fields that had previously been excluded. For example the input line:
details="{"a": { "b": { "c": { "d": 1, "e": 2} } } }"
Would be parsed into these fields: a.b.c.e=2
.
kvParse()
| parseJson(field=details, exclude=a.b.c, include=a.b.c.e)
If includes and excludes are used with prefix, you need to prefix the includes and excludes as well. For example the input line:
details="{"a": { "b": { "c": { "d": 1, "e": 2} } } }"
Would be parsed into these fields: x.a.b.c.e=2
.
kvParse()
| parseJson(field=details, prefix=x., exclude=x.a.b.c, include=x.a.b.c.e)