Skip to content
Content was updated:Jan 28, 2025
This is an archived version of the documentation downloaded from LogScale Documentation Archives. Up to date documentation, and documentation for later versions, is available on LogScale Documentation

## `kvParse()`

Parse events encoded with key/value pairs into individual fields and values. This function can run an extra key/value parser on events.

Parameter | Type | Required | Default Value | Description
--- | --- | --- | --- | ---
`as` | string | optional[a] |  | Prefix for all resolved field keys.
`excludeEmpty` | array of strings | optional[a] | `false` | If the value of a key is empty, exclude the field.
`field`[b] | array of strings | optional[a] | `@rawstring` | Fields that should be key-value parsed.
`override` | boolean | optional[a] | `false` | Override existing values for keys that already exist in the event.
`separator` | string | optional[a] | `=` | The token that separates the key from the value — a single char only.
`separatorPadding` | string | optional[a] | `unknown` | Help the function recognize unquoted empty values and parse them by specifying whether there is a whitespace around the key-value separator (typically `=`). For a list of interpretations, see kvParse() separatorPadding argument handling.
|  |  | Values
|  |  | `no` | Assumes the source does not have a whitespace around the key-value separator, as in `a=1, b=2`
|  |  | `unknown` | Whether the source has a padding (whitespace) around values is not recognized.
|  |  | `yes` | Indicates the source has a whitespace around the key-value separator, as in `a = 1, b = 2`
[a] Optional parameters use their default value unless explicitly set. [b] The parameter name `field` can be omitted.

Hide omitted argument names for this function

Show omitted argument names for this function

> Omitted Argument Names

> The argument name for `field` can be omitted; the following forms of this function are equivalent:

> logscale Syntax

>     kvParse(["value"])

> and:

> logscale Syntax

>     kvParse(field=["value"])

> These examples show basic structure only.

### `kvParse()` Function Operation

The `kvParse()` function is used to parse key/values of the form:

- `key=value`

- `key="value"`

- `key='value'`

- `key = value`

Both key and value can be either quoted using `"` or `'`, or unquoted. If using quotes, the quotes must be terminated.

For a log line like this:

ini

    2017-02-22T13:14:01.917+0000 [main thread] INFO UserService - creating new user id=123, name='john doe' email=john@doe

The key/value parser extracts the fields:

- id=123

- name=john doe

- email=john@doe

Use the parameter `field` to specify which fields should be key/value parsed. Specify @rawstring to key/value parse the rawstring.

### `kvParse()` Syntax Examples

- Key/value parse the log line:

ini

    creating new user id=123, name='john doe' email=john@doe.

This will add the fields id=123, name='john doe' and email=john@doe to the event:

logscale

    kvParse()

- Key/value parse the log line:

ini

    creating new user id=123, name='john doe' email=john@doe loglevel=ERROR.

Assuming the event already has a loglevel field, replacing the value of that field with `ERROR` requires parameter `override=true`:

logscale

    kvParse(override=true)

- Key/value parse a nested field. In this example we will use JSON input:

json

    {
      "service": "paymentService",
      "type": "payment",
      "metadata": "host=server5,transactionID=123,processingTime=100"
    }

and parse out the key/values in the metadata field:

logscale

    parseJson()
    | kvParse(metadata)

- Key/value parse the log line and export fields with a prefix:

ini

    creating new user id=123, name='john doe' email=john@doe.

The following query will add the fields user.id=123, user.name='john doe' and user.email=john@doe to the event:

logscale

    kvParse(as="user")

- Key/value parse the log line:

ini

    firstname = John middlename = lastname = Doe

This will add the fields firstname=John, middleName= (empty value) and lastname=Doe to the event with a whitespace around the key-value separator (`=`):

logscale

    kvParse(separatorPadding="yes")

- When parsing a key/value line, the impact of spacing between the key, value and equals sign can lead to interpretation differences. The `separatorPadding` parameter controls this by defining how different patterns are interpreted with and without spacing, as follows:

  Raw Data | separatorPadding | Field a Value | Field b Value | Notes
  a = b = c | `unknown` | b |  | c dropped
  a = b = c | `yes` | (Empty) | c |
  a = b = c | `no` | (Empty) | (Empty) | c dropped
  a=b=c | `unknown` | b=c |  |
  a=b=c | `yes` | b=c |  |
  a=b=c | `no` | b=c |  |
  a = b=c | `unknown` | b=c |  |
  a = b=c | `yes` | b=c |  |
  a = b=c | `no` | (Empty) | c | c dropped
  a=b = c | `unknown` | b |  |
  a=b = c | `yes` | (Empty) | (Empty) | c dropped
  a=b = c | `no` | b |  |

- Keep the first value for duplicated keys.

  ini

      name='john doe' name='jane doe'

  By default `kvParse()` will keep the last seen value. To keep the first value instead, set as follows:

  logscale

      kvParse(onDuplicate=keepFirst)

  name
  john doe

- Keep the first value for duplicated keys, with a preset field: if name is set to `alice` beforehand, then `onDuplicate=keepFirst` parameter has no effect and name will keep such preset value. If you want to ignore the preset value, use the `override` parameter.

  ini

      name='john doe' name='jane doe'

  logscale

      kvParse(onDuplicate=keepFirst, override=true)

  name
  john doe

- Keep the last value for duplicated keys, with a preset field: if name is set to `alice` beforehand, then the `onDuplicate=Last` parameter has no effect and name will keep such preset value. If you want to ignore the preset value, use the `override` parameter.

  ini

      name='john doe' name='jane doe'

  logscale

      kvParse(onDuplicate=keepLast, override=true)

  name
  jane doe

### `kvParse()` Examples

Click + next to an example below to get the full details.

#### Parse Key-Value Pairs With Override

Extract key-value pairs from text allowing field overrides using the `kvParse()` function with `override` parameter

##### Query

logscale

    kvParse(field=message, override=true)

##### Introduction

In this example, the `kvParse()` function is used with the `override` parameter to extract key-value pairs from a message field, with the ability to override any existing fields that have matching names.

Example incoming data might look like this:

@timestamp | source | status | message
--- | --- | --- | ---
2025-08-06T10:00:00Z | webserver1 | 200 | user=john.doe status=active role=admin sessionID=12345
2025-08-06T10:00:01Z | webserver2 | 404 | user=jane.smith status=inactive role=user sessionID=67890
2025-08-06T10:00:02Z | webserver1 | 500 | user=bob.wilson status=blocked role=guest sessionID=11111
2025-08-06T10:00:03Z | webserver3 | 200 | user=alice.jones status=active role=admin sessionID=22222
2025-08-06T10:00:04Z | webserver2 | 200 | user=mike.brown status=pending role=user sessionID=33333

##### Step-by-Step

1. Starting with the source repository events.

2. logscale

       kvParse(field=message, override=true)

   Parses key-value pairs from the message field containing key-value formatted text.

   The `field` parameter specifies which field contains the key-value pairs to parse. The `override` parameter set to `true` allows the parsed values to overwrite any existing fields with matching names. Without the `override` parameter, the function would skip creating fields that already exist in the event.

3. Event Result set.

##### Summary and Results

The query is used to extract structured fields from key-value formatted text while allowing the new values to replace existing field values.

This query is useful, for example, to parse log messages containing key-value pairs where the extracted values should take precedence over existing fields, or when reprocessing events where field values need to be updated from the message content.

Sample output from the incoming example data:

@timestamp | source | message | user | status | role | sessionID
--- | --- | --- | --- | --- | --- | ---
2025-08-06T10:00:00Z | webserver1 | user=john.doe status=active role=admin sessionID=12345 | john.doe | active | admin | 12345
2025-08-06T10:00:01Z | webserver2 | user=jane.smith status=inactive role=user sessionID=67890 | jane.smith | inactive | user | 67890
2025-08-06T10:00:02Z | webserver1 | user=bob.wilson status=blocked role=guest sessionID=11111 | bob.wilson | blocked | guest | 11111
2025-08-06T10:00:03Z | webserver3 | user=alice.jones status=active role=admin sessionID=22222 | alice.jones | active | admin | 22222
2025-08-06T10:00:04Z | webserver2 | user=mike.brown status=pending role=user sessionID=33333 | mike.brown | pending | user | 33333

Note that the original status field values are overwritten by the parsed values from the message, and new fields (user, role, sessionID) are created from the parsed key-value pairs.

#### Parse String as CSV

Parse a CSV-encoded field into known columns using `parseCsv()` function

##### Query

logscale

    kvParse()
    | parseCsv(result, columns=[count, status,
              completion, precision, sourcetask])

##### Introduction

The `parseCsv()` function can be used to parse a CSV-encoded field into known columns.

Example incoming data might look like this:

Raw Events

2017-02-22T13:14:01.917+0000 [main thread] INFO statsModule got result="117 ,success ,27% ,3.14"

##### Step-by-Step

1. Starting with the source repository events.

2. logscale

       kvParse()

   Parses the string into key value pairs.

3. logscale

       | parseCsv(result, columns=[count, status,
                 completion, precision, sourcetask])

   CSV parses the result field from a log line (extracted by the `kvParse()` function) and adds the following fields to the event: count with the value `117`, status with the value `success`, completion with the value `27%`, and precision with the value `3.14`.

4. Event Result set.

##### Summary and Results

The query is used to parse a string as CSV.

Sample output from the incoming example data:

completion | count | precision | result | status
--- | --- | --- | --- | ---
27% | 117 | 3.14 | 117 ,success ,27% ,3.14 | success

#### Enter search term