parseJson() Query Function

Parses data as JSON. Specify field=@rawstring to parse the rawstring into JSON. It is possible to prefix the names of the extracted fields using the prefix parameter. It is also possible to exclude some of the extracted fields using the exclude parameter, specify exclude=a.b.c to exclude c and all of its descendants or exclude="a.b[*].c" to exclude all c inside the array b. If you need to keep certain descendants of an otherwise excluded path you can use the include parameter.

Parameters

Name

Type

Required

Default

Description

field

string

Yes

@rawstring

Fields that should be parsed as JSON.

prefix

string

No

Prefix the name of the extracted JSON fields with the value of this parameter.

exclude

[string]

No

[]

Fields that should be excluded from the result, supports dot-pathing and array wildcards. If used with prefix the exclude fields should be prefixed as well.

include

[string]

No

[]

Fields that should be included even if they had been previously excluded by use of exclude, supports dot-pathing and array wildcards. If used with prefix the include fields should be prefixed as well.

removePrefixes

[string]

No

[]

Prefixes that should be removed from the names of the extracted JSON fields, supports dot-pathing. If multiple prefixes are supplied, the longest matching prefix will be used.

The implied parameter is field.

Examples

If the whole event sent to Humio is JSON like: .. code-block:: javascript

{“service”: “userService”, “timestamp”: “2017-12-18T20:39:35Z”, “msg”: “user with id=47 logged in”}

humio
parseJson() | parseTimestamp(field=timestamp)

If a field in the incoming event contains JSON like:

ini
2017-12-18T20:39:35Z user id=47 logged in details="{"name": "Peter", "email": "peter@test.com", "id":47}"

In the example below the details field is extracted using the kvparse function and then parseJson is used to parse the JSON inside the details field.

humio
/(?<timestamp>\S+)/ | parseTimestamp(field=timestamp) | kvParse() | parseJson(field=details)

It is possible to prefix names of the extracted JSON fields. This can be useful for avoiding collisions with existing fields with the same name. For example the input line:

humio
added new user details="{"email": "foo@test.com", "name": "Peter"}"

Could be parsed into these fields: user.email=foo@test.com, user.name=Peter.

humio
kvParse() | parseJson(field=details, prefix="user.")

It is possible to remove prefixes as well. For example the input line:

humio
details="{"a": { "b": { "c": { "d": "e", "f": "g"}, "h": "i" }, "j": "k" } }"

Would be parsed into these fields: b.c.d=e, b.c.f=g, b.h=i, j=k.

humio
kvParse() | parseJson(field=details, removePrefixes=a.)

It is possible to exclude extracted fields. This can be useful for removing sensitive data or e.g. large arrays. For example the input line:

humio
details="{"a": { "b": { "c": { "d": "e", "f": "g"}, "h": "i" }, "j": "k" } }"

Would be parsed into these fields: a.b.h=i, a.j=k but not e.g. a.b.c.d=e

humio
kvParse() | parseJson(field=details, exclude=a.b.c)

It is also possible to exclude extracted fields within arrays. For example the input line:

humio
details="{"a": { "b": [{ "c": { "d": 1 }, "e": "f" }, { "c": { "d": 2 }, "e": "h" }] } }"

Would be parsed into these fields: a.b[0].e=f, a.b[1].e=h but not e.g. a.b[0].c.d=1.

humio
kvParse() | parseJson(field=details, exclude="a.b[*].c")

It is possible to include fields that had previously been excluded. For example the input line:

humio
details="{"a": { "b": { "c": { "d": 1, "e": 2} } } }"

Would be parsed into these fields: a.b.c.e=2.

humio
kvParse() | parseJson(field=details, exclude=a.b.c, include=a.b.c.e)

If includes and excludes are used with prefix, you need to prefix the includes and excludes as well. For example the input line:

humio
details="{"a": { "b": { "c": { "d": 1, "e": 2} } } }"

Would be parsed into these fields: x.a.b.c.e=2.

humio
kvParse() | parseJson(field=details, prefix=x., exclude=x.a.b.c, include=x.a.b.c.e)