Parses data as JSON. Specify
field=@rawstring to parse the rawstring
into JSON. It is possible to prefix the names of the extracted fields
using the prefix parameter. It is also possible to exclude some of the
extracted fields using the exclude parameter, specify
exclude=a.b.c to exclude
c and all of its descendants or
exclude="a.b[*].c" to exclude all
c inside the array
b. If you need to keep certain descendants
of an otherwise excluded path you can use the include parameter.
Function Traits: Transformation
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
exclude | Array of strings | optional | [] | Fields that should be excluded from the result, supports dot-pathing and array wildcards. If used with prefix the exclude fields should be prefixed as well. |
excludeEmpty | Array of strings | optional | false | Whether to exclude if the field is empty |
field[a] | string | required | @rawstring | Fields that should be parsed as JSON. |
handleNull | Array of strings | optional | keep | How to handle null values: keep; empty; discard. |
include | Array of strings | optional | [] | Fields that should be included even if they had been previously excluded by use of exclude, supports dot-pathing and array wildcards. If used with prefix the include fields should be prefixed as well. |
prefix | string | optional | blank | Prefix the name of the extracted JSON fields with the value of this parameter. |
removePrefixes | Array of strings | optional | [] | Prefixes that should be removed from the names of the extracted JSON fields, supports dot-pathing. If multiple prefixes are supplied, the longest matching prefix will be used. |
The parameter name for field can be omitted; the following forms are equivalent:
parseJson("@rawstring")and:
parseJson(field="@rawstring")parseJson() Examples
If the whole event sent to LogScale is JSON like:
{"service": "userService", "timestamp": "2017-12-18T20:39:35Z", "msg": "user with id=47 logged in"}parseJson()
| parseTimestamp(field=timestamp)If a field in the incoming event contains JSON like:
2017-12-18T20:39:35Z user id=47 logged in details="{"name": "Peter", "email": "peter@test.com", "id":47}"In the example below the details field is extracted using the kvparse function and then parseJson is used to parse the JSON inside the details field.
/(?<timestamp>\S+)/
| parseTimestamp(field=timestamp)
| kvParse()
| parseJson(field=details)It is possible to prefix names of the extracted JSON fields. This can be useful for avoiding collisions with existing fields with the same name. For example the input line:
added new user details="{"email": "foo@test.com", "name": "Peter"}"
Could be parsed into these fields:
<user.email=foo@test.com>, user.name=Peter.
kvParse()
| parseJson(field=details, prefix="user.")It is possible to remove prefixes as well. For example the input line:
details="{"a": { "b": { "c": { "d": "e", "f": "g"}, "h": "i" }, "j": "k" } }"Would be parsed into these fields: b.c.d=e, b.c.f=g, b.h=i, j=k.
kvParse()
| parseJson(field=details, removePrefixes=a.)It is possible to exclude extracted fields. This can be useful for removing sensitive data or e.g. large arrays. For example the input line:
details="{"a": { "b": { "c": { "d": "e", "f": "g"}, "h": "i" }, "j": "k" } }"Would be parsed into these fields: a.b.h=i, a.j=k but not e.g. a.b.c.d=e
kvParse()
| parseJson(field=details, exclude=a.b.c)It is also possible to exclude extracted fields within arrays. For example the input line:
details="{"a": { "b": [{ "c": { "d": 1 }, "e": "f" }, { "c": { "d": 2 }, "e": "h" }] } }"Would be parsed into these fields: a.b[0].e=f, a.b[1].e=h but not e.g. a.b[0].c.d=1.
kvParse()
| parseJson(field=details, exclude="a.b[*].c")It is possible to include fields that had previously been excluded. For example the input line:
details="{"a": { "b": { "c": { "d": 1, "e": 2} } } }"Would be parsed into these fields: a.b.c.e=2.
kvParse()
| parseJson(field=details, exclude=a.b.c, include=a.b.c.e)If includes and excludes are used with prefix, you need to prefix the includes and excludes as well. For example the input line:
details="{"a": { "b": { "c": { "d": 1, "e": 2} } } }"Would be parsed into these fields: x.a.b.c.e=2.
kvParse()
| parseJson(field=details, prefix=x., exclude=x.a.b.c, include=x.a.b.c.e)