Skip to content
This is an archived version of the documentation downloaded from LogScale Documentation Archives. Up to date documentation, and documentation for later versions, is available on LogScale Documentation

## `parseJson()`

Parses data or a field as JSON, converting the data into named fields and arrays within the event. Use `parseJson(field=@rawstring)` to parse the original ingested rawstring into JSON. Use the `prefix` parameter to prefix the names of the extracted fields.

Furthermore, to exclude some of the extracted fields, use the `exclude` parameter to specify the JSON object structure to be excluded. For example, use `parseJson(field=input,exclude=a.b.c)` to exclude `c` and all of its descendants (a path-based exclusion) or use `parseJson(field=input,exclude="a.b[*].c")` to exclude all `c` inside the array `b` (array-based exclusion). Note that all non-nested fields are also excluded.

Use the `include` parameter if you need to keep certain descendants of an otherwise excluded path or an excluded non-nested field in the output.

Parameter | Type | Required | Default Value | Description
--- | --- | --- | --- | ---
`exclude` | array of strings | optional[a] | `[]` | Removes specified JSON fields from the output. Use on fields that should be excluded from the result. Supports dot-pathing and array wildcards. If used with prefix, the excluded fields will use the specified prefix. The exclusion works as a wildcard match for the given field; for example, the value `query` will match both nested fields (like query.string) and similarly non-nested named fields (like queryString, queryStart, and queryEnd). To retain specific fields while excluding others, use the `include` parameter.
`excludeEmpty` | boolean | optional[a] | `false` | Whether to exclude if the field is empty.
|  |  | Values
|  |  | `false` | Do not exclude the field, even if the value is empty
|  |  | `true` | Exclude the field if the value is empty
`field`[b] | string | required | `@rawstring` | Fields that should be parsed as JSON.
`handleNull` | string | optional[a] | `keep` | How null values are handled.
|  |  | Values
|  |  | `discard` | Discard the null value and field null value with an empty string `""`
|  |  | `empty` | Replaces a null value with an empty string `""`
|  |  | `keep` | Converts the value to the `"null"` string
`include` | array of strings | optional[a] | `[]` | Retains specific descendants of excluded paths. Use on fields that should be included, even if they had been previously excluded by use of `exclude`. Supports dot-pathing and array wildcards. If used with `prefix`, the include fields will also use the specified prefix.
`prefix` | string | optional[a] | `blank` | Prefix the name of the extracted JSON fields with the value of this parameter.
`removePrefixes` | array of strings | optional[a] | `[]` | Prefixes that should be removed from the names of the extracted JSON fields; supports dot-pathing. If multiple prefixes are supplied, the longest matching prefix will be used.
[a] Optional parameters use their default value unless explicitly set. [b] The parameter name `field` can be omitted.

Hide omitted argument names for this function

Show omitted argument names for this function

> Omitted Argument Names

> The argument name for `field` can be omitted; the following forms of this function are equivalent:

> logscale Syntax

>     parseJson("value")

> and:

> logscale Syntax

>     parseJson(field="value")

> These examples show basic structure only.

### `parseJson()` Function Operation

When parsing JSON, the following apply:

- When `excludeEmpty=true` is used, the key/value pairs will be discarded completely whenever the original json contains `foo: ""`

- When the JSON contains `foo: null`, using `handleNull=discard` then the entire key/value pair is discarded, regardless of the setting for `excludeEmpty`

### `parseJson()` Syntax Examples

If the whole event sent to LogScale is JSON like:

javascript

    {"service": "userService", "timestamp": "2017-12-18T20:39:35Z", "msg": "user with id=47 logged in"}

logscale

    parseJson()
    | parseTimestamp(field=timestamp)

If a field in the incoming event contains JSON like:

Raw Events

2017-12-18T20:39:35Z user id=47 logged in details="{"name": "Peter", "email": "peter@test.com", "id":47}"

In the example below the details field is extracted using the `kvParse()` function and then parseJson is used to parse the JSON inside the details field.

logscale

    /(?<timestamp>\S+)/
    | parseTimestamp(field=timestamp)
    | kvParse()
    | parseJson(field=details)

It is possible to prefix names of the extracted JSON fields. This can be useful for avoiding collisions with existing fields with the same name. For example the input line:

logscale

    details="{"email": "foo@test.com", "name": "Peter"}"

Could be parsed into these fields: `user.email=foo@test.com`, `user.name=Peter`.

logscale

    kvParse()
    | parseJson(field=details, prefix="user.")

It is possible to remove prefixes as well. For example the input line:

logscale

    details="{"a": { "b": { "c": { "d": "e", "f": "g"}, "h": "i" }, "j": "k" } }"

If the JSON object is expanded:

json

    {
       "a" : {
          "b" : {
             "c" : {
                "d" : "e",
                "f" : "g"
             },
             "h" : "i"
          },
          "j" : "k"
       }
    }

Would be parsed into these fields with the following function if the `a` prefix is removed: `b.c.d=e`, `b.c.f=g`, `b.h=i`, `j=k`.

logscale

    kvParse()
    | parseJson(field=details, removePrefixes=a.)

It is possible to exclude extracted fields. This can be useful for removing sensitive data or, for example, large arrays. For example, the input line:

logscale

    details="{"a": { "b": { "c": { "d": "e", "f": "g"}, "h": "i" }, "j": "k" } }"

If the raw JSON is formatted:

json

    {
       "a" : {
          "b" : {
             "c" : {
                "d" : "e",
                "f" : "g"
             },
             "h" : "i"
          },
          "j" : "k"
       }
    }

The JSON string would be parsed into these fields: `a.b.h=i`, `a.j=k` but not, for example, `a.b.c.d=e`

logscale

    kvParse()
    | parseJson(field=details, exclude=a.b.c)

It is also possible to exclude extracted fields within arrays. For example the input line:

logscale

    details="{"a": { "b": [{ "c": { "d": 1 }, "e": "f" }, { "c": { "d": 2 }, "e": "h" }] } }"

As a formatted JSON string:

json

    {
       "a" : {
          "b" : [
             {
                "c" : {
                   "d" : 1
                },
                "e" : "f"
             },
             {
                "c" : {
                   "d" : 2
                },
                "e" : "h"
             }
          ]
       }
    }

Would be parsed into these fields: `a.b[0].e=f`, `a.b[1].e=h` but not, for example, `a.b[0].c.d=1`.

logscale

    kvParse()
    | parseJson(field=details, exclude="a.b[*].c")

It is possible to include fields that had previously been excluded. For example the input line:

logscale

    details="{"a": { "b": { "c": { "d": 1, "e": 2} } } }"

Would be parsed into these fields: `a.b.c.e=2`.

logscale

    kvParse()
    | parseJson(field=details, exclude=a.b.c, include=a.b.c.e)

If includes and excludes are used with prefix, you need to prefix the includes and excludes as well. For example the input line:

logscale

    details="{"a": { "b": { "c": { "d": 1, "e": 2} } } }"

Would be parsed into these fields: `x.a.b.c.e=2`.

logscale

    kvParse()
    | parseJson(field=details, prefix=x., exclude=x.a.b.c, include=x.a.b.c.e)

### `parseJson()` Examples

Click + next to an example below to get the full details.

#### Create Two Temporary Events for Troubleshooting - Example 2

Create two temporary events for testing or troubleshooting using the `createEvents()` function with `parseJson()`

##### Query

logscale

    createEvents(["{\"animal\":{\"kind\":\"dog\", \"weight\":7.0}}", "{\"animal\":{\"kind\":\"cat\", \"weight\":4.2}}"])
    | parseJson()

##### Introduction

In this example, the `createEvents()` function is combined with `parseJson()` to parse @rawstring as JSON.

Example incoming data might look like this:

json

    [
    {"animal":{"kind":"dog", "weight":7.0}},
    {"animal":{"kind":"cat", "weight":4.2}}
    ]

##### Step-by-Step

1. Starting with the source repository events.

2. logscale

       createEvents(["{\"animal\":{\"kind\":\"dog\", \"weight\":7.0}}", "{\"animal\":{\"kind\":\"cat\", \"weight\":4.2}}"])

   Creates two temporary events. An event with `dog` and an event with `cat`.

3. logscale

       | parseJson()

   Parses specified fields as JSON.

4. Event Result set.

##### Summary and Results

The query is used to create temporary events and parse the @rawstring as JSON.

Sample output from the incoming example data:

@timestamp | animal.kind | animal.weight
--- | --- | ---
1733311547717 | dog | 7.0
1733311547717 | cat | 4.2

#### Create Two Temporary Events for Troubleshooting - Example 3

Create two temporary events for testing or troubleshooting using the `createEvents()` function with `kvParse()`

##### Query

logscale

    createEvents(["animal=dog weight=7.0", "animal=cat weight=4.2"])
    | kvParse()

##### Introduction

In this example, the `createEvents()` function is combined with `kvParse()` to parse @rawstring as JSON.

Example incoming data might look like this:

animal=dog weight=7.0
animal=cat weight=4.2

##### Step-by-Step

1. Starting with the source repository events.

2. logscale

       createEvents(["animal=dog weight=7.0", "animal=cat weight=4.2"])

   Creates two temporary events. An event with `dog` and an event with `cat`.

3. logscale

       | kvParse()

   Parses the string into key value pairs.

4. Event Result set.

##### Summary and Results

The query is used to create temporary events and parse the @rawstring as key value pairs.

Sample output from the incoming example data:

animal | weight
--- | ---
dog | 7.0
cat | 4.2

#### Parse JSON Content With Specific Parameters

Parse JSON content with specific parameters while excluding the actual query content using the `parseJson()`

##### Query

logscale

    #type=audit-log
    | /"type":"alert.update"/
    | parseJson(exclude="query", include="queryStart")

##### Introduction

In this example, the `parseJson()` function is used to search audit logs for alert update events, specifically looking at the queryStart field (timestamp), while excluding the actual query content (query.xxx).

##### Step-by-Step

1. Starting with the source repository events.

2. logscale

       #type=audit-log

   Filters for all events from repository `audit-log`.

3. logscale

       | /"type":"alert.update"/

   Filters for events (audit logs) where the @rawstring field contains the string `/"type":"alert.update"/`.

4. logscale

       | parseJson(exclude="query", include="queryStart")

   With specific parameters set, parses the JSON content excluding the query field and including only the queryStart field.

   The exclusion works as a wildcard match for the given field; for example, the value `query` will match both nested fields (like query.string) and similarly non-nested named fields (like queryString, queryStart, and queryEnd). In this example, to retain the specific non-nested field queryStart while excluding the others, the `include` parameter is used.

5. Event Result set.

##### Summary and Results

The query is used to search audit logs for alert update events, specifically looking at the non-nested queryStart field, while excluding the actual query content (query.xxx). The query is useful if, for example, you want to review alert activity without the overhead of full query contents, track temporal patterns in alert updates, or investigate alert timing issues further.

#### Enter search term