Parses data as JSON. Specify field=@rawstring to parse the rawstring into JSON. It is possible to prefix the names of the extracted fields using the prefix parameter. It is also possible to exclude some of the extracted fields using the exclude parameter, specify exclude=a.b.c to exclude c and all of its descendants or exclude="a.b[*].c" to exclude all c inside the array b. If you need to keep certain descendants of an otherwise excluded path you can use the include parameter.

ParameterTypeRequiredDefault ValueDescription
excludeArray of stringsoptional[a][] Fields that should be excluded from the result, supports dot-pathing and array wildcards. If used with prefix the exclude fields should be prefixed as well.
excludeEmptybooleanoptional[a]false Whether to exclude if the field is empty.
   Valid Values
   falseDon't exclude the field, even if the value is empty
   trueExclude the field if the value is empty
field[b]stringrequired@rawstring Fields that should be parsed as JSON.
handleNullstringoptional[a]keep How null values are handled.
   Valid Values
   discardDiscard the null value and field null value with an empty string ""
   emptyReplaces a null value with an empty string ""
   keepConverts the value to the "null" string
includeArray of stringsoptional[a][] Fields that should be included even if they had been previously excluded by use of exclude, supports dot-pathing and array wildcards. If used with prefix the include fields should be prefixed as well.
prefixstringoptional[a]blank Prefix the name of the extracted JSON fields with the value of this parameter.
removePrefixesArray of stringsoptional[a][] Prefixes that should be removed from the names of the extracted JSON fields, supports dot-pathing. If multiple prefixes are supplied, the longest matching prefix will be used.

[a] Optional parameters use their default value unless explicitly set.

[b] The argument name field can be omitted.

Hide omitted argument names for this function

Show omitted argument names for this function

When parsing JSON, the following apply:

  • When excludeEmpty=true is used, the key/value pairs will be discarded completely whenever the original json contains foo: ""

  • When the JSON contains foo: null, using handleNull=discard then the entire key/value pair is discarded, regardless of the setting for excludeEmpty

parseJson() Examples

If the whole event sent to LogScale is JSON like:

javascript
{"service": "userService", "timestamp": "2017-12-18T20:39:35Z", "msg": "user with id=47 logged in"}
logscale
parseJson()
| parseTimestamp(field=timestamp)

If a field in the incoming event contains JSON like:

ini
2017-12-18T20:39:35Z user id=47 logged in details="{"name": "Peter", "email": "peter@test.com", "id":47}"

In the example below the details field is extracted using the kvparse function and then parseJson is used to parse the JSON inside the details field.

logscale
/(?<timestamp>\S+)/
| parseTimestamp(field=timestamp)
| kvParse()
| parseJson(field=details)

It is possible to prefix names of the extracted JSON fields. This can be useful for avoiding collisions with existing fields with the same name. For example the input line:

logscale
added new user details="{"email": "foo@test.com", "name": "Peter"}"

Could be parsed into these fields: , user.name=Peter.

logscale
kvParse()
| parseJson(field=details, prefix="user.")

It is possible to remove prefixes as well. For example the input line:

logscale
details="{"a": { "b": { "c": { "d": "e", "f": "g"}, "h": "i" }, "j": "k" } }"

If the JSON object is expanded:

json
{
   "a" : {
      "b" : {
         "c" : {
            "d" : "e",
            "f" : "g"
         },
         "h" : "i"
      },
      "j" : "k"
   }
}

Would be parsed into these fields with the following function if the a prefix is removed: b.c.d=e, b.c.f=g, b.h=i, j=k.

logscale
kvParse()
| parseJson(field=details, removePrefixes=a.)

It is possible to exclude extracted fields. This can be useful for removing sensitive data or, for example, large arrays. For example, the input line:

logscale
details="{"a": { "b": { "c": { "d": "e", "f": "g"}, "h": "i" }, "j": "k" } }"

If the raw JSON is formatted:

json
{
   "a" : {
      "b" : {
         "c" : {
            "d" : "e",
            "f" : "g"
         },
         "h" : "i"
      },
      "j" : "k"
   }
}

The JSON string would be parsed into these fields: a.b.h=i, a.j=k but not, for example, a.b.c.d=e

logscale
kvParse()
| parseJson(field=details, exclude=a.b.c)

It is also possible to exclude extracted fields within arrays. For example the input line:

logscale
details="{"a": { "b": [{ "c": { "d": 1 }, "e": "f" }, { "c": { "d": 2 }, "e": "h" }] } }"

As a formatted JSON string:

json
{
   "a" : {
      "b" : [
         {
            "c" : {
               "d" : 1
            },
            "e" : "f"
         },
         {
            "c" : {
               "d" : 2
            },
            "e" : "h"
         }
      ]
   }
}

Would be parsed into these fields: a.b[0].e=f, a.b[1].e=h but not, for example, a.b[0].c.d=1.

logscale
kvParse()
| parseJson(field=details, exclude="a.b[*].c")

It is possible to include fields that had previously been excluded. For example the input line:

logscale
details="{"a": { "b": { "c": { "d": 1, "e": 2} } } }"

Would be parsed into these fields: a.b.c.e=2.

logscale
kvParse()
| parseJson(field=details, exclude=a.b.c, include=a.b.c.e)

If includes and excludes are used with prefix, you need to prefix the includes and excludes as well. For example the input line:

logscale
details="{"a": { "b": { "c": { "d": 1, "e": 2} } } }"

Would be parsed into these fields: x.a.b.c.e=2.

logscale
kvParse()
| parseJson(field=details, prefix=x., exclude=x.a.b.c, include=x.a.b.c.e)