Parses data or a field as JSON, converting the data into named fields and arrays within the event. Use parseJson(field=@rawstring) to parse the original ingested rawstring into JSON. Use the prefix parameter to prefix the names of the extracted fields.

Furthermore, to exclude some of the extracted fields, use the exclude parameter to specify the JSON object structure to be excluded. For example, use parseJson(field=input,exclude=a.b.c) to exclude c and all of its descendants (a path-based exclusion) or use parseJson(field=input,exclude="a.b[*].c") to exclude all c inside the array b (array-based exclusion). Note that all non-nested fields are also excluded.

Use the include parameter if you need to keep certain descendants of an otherwise excluded path or an excluded non-nested field in the output.

ParameterTypeRequiredDefault ValueDescription
excludearray of stringsoptional[a] [] Removes specified JSON fields from the output. Use on fields that should be excluded from the result. Supports dot-pathing and array wildcards. If used with prefix, the excluded fields will use the specified prefix. The exclusion works as a wildcard match for the given field; for example, the value query will match both nested fields (like query.string) and similarly non-nested named fields (like queryString, queryStart, and queryEnd). To retain specific fields while excluding others, use the include parameter.
excludeEmptybooleanoptional[a] false Whether to exclude if the field is empty.
   Values
   falseDo not exclude the field, even if the value is empty
   trueExclude the field if the value is empty
field[b]stringrequired @rawstring Fields that should be parsed as JSON.
handleNullstringoptional[a] keep How null values are handled.
   Values
   discardDiscard the null value and field null value with an empty string ""
   emptyReplaces a null value with an empty string ""
   keepConverts the value to the "null" string
includearray of stringsoptional[a] [] Retains specific descendants of excluded paths. Use on fields that should be included, even if they had been previously excluded by use of exclude. Supports dot-pathing and array wildcards. If used with prefix, the include fields will also use the specified prefix.
prefixstringoptional[a] blank Prefix the name of the extracted JSON fields with the value of this parameter.
removePrefixesarray of stringsoptional[a] [] Prefixes that should be removed from the names of the extracted JSON fields; supports dot-pathing. If multiple prefixes are supplied, the longest matching prefix will be used.

[a] Optional parameters use their default value unless explicitly set.

[b] The parameter name field can be omitted.

Hide omitted argument names for this function

Show omitted argument names for this function

When parsing JSON, the following apply:

  • When excludeEmpty=true is used, the key/value pairs will be discarded completely whenever the original json contains foo: ""

  • When the JSON contains foo: null, using handleNull=discard then the entire key/value pair is discarded, regardless of the setting for excludeEmpty

parseJson() Syntax Examples

If the whole event sent to LogScale is JSON like:

javascript
{"service": "userService", "timestamp": "2017-12-18T20:39:35Z", "msg": "user with id=47 logged in"}
logscale
parseJson()
| parseTimestamp(field=timestamp)

If a field in the incoming event contains JSON like:

ini
2017-12-18T20:39:35Z user id=47 logged in details="{"name": "Peter", "email": "peter@test.com", "id":47}"

In the example below the details field is extracted using the kvParse() function and then parseJson is used to parse the JSON inside the details field.

logscale
/(?<timestamp>\S+)/
| parseTimestamp(field=timestamp)
| kvParse()
| parseJson(field=details)

It is possible to prefix names of the extracted JSON fields. This can be useful for avoiding collisions with existing fields with the same name. For example the input line:

logscale
details="{"email": "foo@test.com", "name": "Peter"}"

Could be parsed into these fields: user.email=foo@test.com, user.name=Peter.

logscale
kvParse()
| parseJson(field=details, prefix="user.")

It is possible to remove prefixes as well. For example the input line:

logscale
details="{"a": { "b": { "c": { "d": "e", "f": "g"}, "h": "i" }, "j": "k" } }"

If the JSON object is expanded:

json
{
   "a" : {
      "b" : {
         "c" : {
            "d" : "e",
            "f" : "g"
         },
         "h" : "i"
      },
      "j" : "k"
   }
}

Would be parsed into these fields with the following function if the a prefix is removed: b.c.d=e, b.c.f=g, b.h=i, j=k.

logscale
kvParse()
| parseJson(field=details, removePrefixes=a.)

It is possible to exclude extracted fields. This can be useful for removing sensitive data or, for example, large arrays. For example, the input line:

logscale
details="{"a": { "b": { "c": { "d": "e", "f": "g"}, "h": "i" }, "j": "k" } }"

If the raw JSON is formatted:

json
{
   "a" : {
      "b" : {
         "c" : {
            "d" : "e",
            "f" : "g"
         },
         "h" : "i"
      },
      "j" : "k"
   }
}

The JSON string would be parsed into these fields: a.b.h=i, a.j=k but not, for example, a.b.c.d=e

logscale
kvParse()
| parseJson(field=details, exclude=a.b.c)

It is also possible to exclude extracted fields within arrays. For example the input line:

logscale
details="{"a": { "b": [{ "c": { "d": 1 }, "e": "f" }, { "c": { "d": 2 }, "e": "h" }] } }"

As a formatted JSON string:

json
{
   "a" : {
      "b" : [
         {
            "c" : {
               "d" : 1
            },
            "e" : "f"
         },
         {
            "c" : {
               "d" : 2
            },
            "e" : "h"
         }
      ]
   }
}

Would be parsed into these fields: a.b[0].e=f, a.b[1].e=h but not, for example, a.b[0].c.d=1.

logscale
kvParse()
| parseJson(field=details, exclude="a.b[*].c")

It is possible to include fields that had previously been excluded. For example the input line:

logscale
details="{"a": { "b": { "c": { "d": 1, "e": 2} } } }"

Would be parsed into these fields: a.b.c.e=2.

logscale
kvParse()
| parseJson(field=details, exclude=a.b.c, include=a.b.c.e)

If includes and excludes are used with prefix, you need to prefix the includes and excludes as well. For example the input line:

logscale
details="{"a": { "b": { "c": { "d": 1, "e": 2} } } }"

Would be parsed into these fields: x.a.b.c.e=2.

logscale
kvParse()
| parseJson(field=details, prefix=x., exclude=x.a.b.c, include=x.a.b.c.e)

parseJson() Examples

Click + next to an example below to get the full details.

Parse JSON Content With Specific Parameters

Parse JSON content with specific parameters while excluding the actual query content using the parseJson()

Query
logscale
#type=audit-log
| /"type":"alert.update"/
| parseJson(exclude="query", include="queryStart")
Introduction

In this example, the parseJson() function is used to search audit logs for alert update events, specifically looking at the queryStart field (timestamp), while excluding the actual query content (query.xxx).

Step-by-Step

  1. Starting with the source repository events.

  2. logscale
    #type=audit-log

    Filters for all events from repository audit-log.

  3. logscale
    | /"type":"alert.update"/

    Filters for events (audit logs) where the @rawstring field contains the string /"type":"alert.update"/.

  4. logscale
    | parseJson(exclude="query", include="queryStart")

    With specific parameters set, parses the JSON content excluding the query field and including only the queryStart field.

    The exclusion works as a wildcard match for the given field; for example, the value query will match both nested fields (like query.string) and similarly non-nested named fields (like queryString, queryStart, and queryEnd). In this example, to retain the specific non-nested field queryStart while excluding the others, the include parameter is used.

  5. Event Result set.

Summary and Results

The query is used to search audit logs for alert update events, specifically looking at the non-nested queryStart field, while excluding the actual query content (query.xxx). The query is useful if, for example, you want to review alert activity without the overhead of full query contents, track temporal patterns in alert updates, or investigate alert timing issues further.

Create Two Temporary Events for Troubleshooting - Example 2

Create two temporary events for testing or troubleshooting using the createEvents() function with parseJson()

Query
logscale
createEvents(["{\"animal\":{\"kind\":\"dog\", \"weight\":7.0}}", "{\"animal\":{\"kind\":\"cat\", \"weight\":4.2}}"])
| parseJson()
Introduction

In this example, the createEvents() function is combined with parseJson() to parse @rawstring as JSON.

Example incoming data might look like this:

animal
HASH(0x56552c0bb078)
HASH(0x56552b80d538)
Step-by-Step

  1. Starting with the source repository events.

  2. logscale
    createEvents(["{\"animal\":{\"kind\":\"dog\", \"weight\":7.0}}", "{\"animal\":{\"kind\":\"cat\", \"weight\":4.2}}"])

    Creates two temporary events. An event with dog and an event with cat.

  3. logscale
    | parseJson()

    Parses specified fields as JSON.

  4. Event Result set.

Summary and Results

The query is used to create temporary events and parse the @rawstring as JSON.

Sample output from the incoming example data:

@rawstring@timestamp@timestamp.nanosanimal.kindanimal.weight
{"animal":{"kind":"dog", "weight":7.0}}17333115477170dog7.0
{"animal":{"kind":"cat", "weight":4.2}}17333115477170cat4.2

Create Two Temporary Events for Troubleshooting - Example 3

Create two temporary events for testing or troubleshooting using the createEvents() function with kvParse()

Query
logscale
createEvents(["animal=dog weight=7.0", "animal=cat weight=4.2"])
| kvParse()
Introduction

In this example, the createEvents() function is combined with kvParse() to parse @rawstring as JSON.

Example incoming data might look like this:

animal=dog weight=7.0
animal=cat weight=4.2
Step-by-Step

  1. Starting with the source repository events.

  2. logscale
    createEvents(["animal=dog weight=7.0", "animal=cat weight=4.2"])

    Creates two temporary events. An event with dog and an event with cat.

  3. logscale
    | kvParse()

    Parses the string into key value pairs.

  4. Event Result set.

Summary and Results

The query is used to create temporary events and parse the @rawstring as key value pairs.

Sample output from the incoming example data:

animalweight
dog7.0
cat4.2