Skip to content
Content was updated: Nov 12, 2024
This is an archived version of the documentation downloaded from LogScale Documentation Archives. Up to date documentation, and documentation for later versions, is available on LogScale Documentation

## Removing Fields

The documentation explains how to remove fields during data ingestion in LogScale to optimize costs, specifically for structured data and metadata fields that were defined outside of the @rawstring. While this feature helps manage ingest license usage, it has specific limitations - it cannot modify the original @rawstring or remove fields extracted from it, and the removal process must occur before events are parsed.

### Important

The ability to remove fields during parsing is designed to assist in cost optimization when ingesting data.

Where data has been ingested with defined fields (i.e., the event information was submitted as structured data), the metadata fields can be removed from the incoming events during ingest, before parsing.

This feature cannot be used in the following scenarios:

- Cannot be used to change or remove the original @rawstring

- Cannot remove fields extracted from @rawstring

- Can only remove fields that were ingested outside the @rawstring, i.e. from structured data or metadata.

For example, when data has been ingested through the API using the following payload:

json

    [
      {
        "fields": {
          "host": "webhost1"
        },
        "messages": [
          "192.168.1.21 - user1 [02/Nov/2017:13:48:26 +0000] \"POST /humio/api/v1/ingest/elastic-bulk HTTP/1.1\" 200 0 \"-\" \"useragent\" 0.015 664 0.015",
          "192.168.1.49 - user1 [02/Nov/2017:13:48:33 +0000] \"POST /humio/api/v1/ingest/elastic-bulk HTTP/1.1\" 200 0 \"-\" \"useragent\" 0.014 657 0.014",
          "192.168.1..21 - user2 [02/Nov/2017:13:49:09 +0000] \"POST /humio/api/v1/ingest/elastic-bulk HTTP/1.1\" 200 0 \"-\" \"useragent\" 0.013 565 0.013",
          "192.168.1.54 - user1 [02/Nov/2017:13:49:10 +0000] \"POST /humio/api/v1/ingest/elastic-bulk HTTP/1.1\" 200 0 \"-\" \"useragent\" 0.015 650 0.015"
        ]
      }
    ]

Only the defined field, host can be removed using this method. The @rawstring and any fields parsed or extracted from the @rawstring cannot be removed using this method.

When removing fields using this method, the fields specified will be removed before events are parsed. Removed fields will not count as ingest towards your license. See Ingest Usage Management for more information on how ingest is measured.

1. Go to your repository and click Parsers.

2. On the `Parsers` page, click on the required parser. The Parser script editor is displayed.

3. Click Settings next to Code, then click Fields to remove in the side menu.

   On the `Fields to remove` page, enter the name of the field to remove and click Save.

   Figure 50. Remove Fields

To remove data or fields during ingest that cannot be removed using this method, use the `replace()` to modify the incoming @rawstring.

#### Enter search term