CrowdStrike Parsing Standard (CPS) 1.0

Following CrowdStrike Parsing Standard (CPS) helps you ingest data in a way that simplifies writing queries that combine data across different data sources. This page provides you with an introduction on how to write parsers that conform to this standard, focusing on normalization.

The majority of the parsers on Falcon LogScale Package Marketplace follow this standard. For overall guidance on writing parsers in LogScale, see Creating a Parser and Developer Guidelines for Packages.

What is normalization in LogScale?

Extracting fields and tags during the parsing stage is crucial for search performance. Mapping different vendor field names (for example timestamp or IP) during the parsing stage to a unified field name helps you simplify your queries. Instead of switching between different fields that contain common information, for example ip_source and source-ip depending on the log format, you will be able to use consistent field names across different vendors and different log formats.

Why CrowdStrike Parsing Standard (CPS)

The Crowdstrike Parsing Standard builds on the Elastic Common Schema (ECS) 8.x. ECS isn't specific to any data store, which provides a lot of flexibility. It's a mature and proven common schema for metrics, logs, traces and resources, managed by the OpenTelemetry community which shares our interest in the convergence of observability and security. See ECS Categorization fields for more detail on ECS fields.

CPS differs from ECS in a number of ways that build on the specifics of LogScale core architecture. For example, parsers that follow CPS make all fields in a log event available as actual LogScale fields, even if they don't match a field in ECS. See Variations to the ECS for more details on the differences between ECS and CPS.

For any given data source, when developing a parser, you can decide which domain specific fields are applicable to the data.

To create CPS compliant Parsers:

  1. Use CPS Compliant Parser tags:

    • #Cps.version

    • #Vendor

    • #ecs.version

    • #event.dataset

    • #event.kind

    • #event.module

    • #event.outcome

    • #observer.type

  2. Your parsers must populate the fields described in Fields Expected From CPS Compliant Parsers

  3. Manage any fields which do not match the ECS fields, see Managing Non ECS Fields.

  4. Add any fields you need to the standard, see Adding Fields to the Standard.

Fields Expected From CPS Compliant Parsers

The fields in the table below define the CPS compliant fields expected to be created during parsing:

Fields Description

Event categorization fields (kind, type, category, outcome)

  • event.outcome is only assigned when an event can logically contain an outcome.

  • event.type and event.category are assigned as LogScale arrays. The arrays can be empty.

ecs.version

This field contains the version of ECS that is being followed by the parser.

Cps.version

  • This field contains a MAJOR.MINOR.PATCH version number following Semantic Versioning.

  • The version denotes the version of this standard, which was targeted by the parser during ingest.

Parser.version

  • This field contains a MAJOR.MINOR.PATCH version number following Semantic Versioning.

  • This version number is specific to the parser which parsed the event, and not related to e.g. the version of the package the parser may have been installed from.

  • The rules for updating the version number are:

    • Any change to an existing field (large or small) is a breaking change, and requires a new major version.

    • If new fields are added, then a new minor version is usually sufficient.

    • Patch versions are for parser changes that do not affect which fields are output by it (performance optimization, etc.)

Vendor

  • If the event was parsed with a parser from a package, the vendor name used here must match the vendor name used in the package scope (e.g. fortinet for fortinet/fortigate).

  • If a parser sets any of the following fields, those must be consistent with vendor names used in other CPS-compliant parsers:

    • observer.vendor

    • vulnerability.scanner.vendor

    • device.manufacturer

event.module Contains roughly the name of the product or service that the event belongs to. Existing event.module values is reused whenever appropriate.
event.dataset

  • Contains the specific name of the dataset within the module described by event.module, prefixed by the value of event.module with a dot in between.

  • If it wouldn't contain any information beyond what is present in event.module, the parser doesn't create this field.

    After ingest all the fields in LogScale are prefixed with #.

    Example combinations of the above fields in the parser can be:

    Vendor event.module event.dataset
    microsoft azure azure.entraid
    zscaler zia zia.web

Managing Non ECS Fields

CPS compliant parsers strive to make all fields in a log event available as actual LogScale fields, even if they don't match a field in ECS:

  • If the event contains fields which don't exist in ECS, their name is prefixed with the string literal Vendor.

    This gives the ECS fields the root namespace, while vendor specific fields can always be found with the Vendor. prefix.

  • If a field can exist as both an ECS field and a vendor specific field, the following logic applies:

    • If the value of both fields is byte-for-byte the same, it only preserves the ECS field, and discards the vendor specific field.

    • If the value of the fields differ, both fields are preserved. For example, an ECS field may require its value be lower cased, but the original log has mixed casing. In that case, the vendor specific field contains the original, mixed case value.

Adding Fields to the Standard

When adding new fields to this standard, all fields which are not taken directly from ECS must have a capital letter from the point where they differ from the schema.

  • Using capital letters for field names follows the guidance from ECS on how to add event fields outside the schema.

  • Example of fully custom field: Parser.version is similar to ecs.version, but the Parser namespace is our own, so must start with a capital letter.

  • Example of extending ECS with custom field: observer.Fictional_field where observer is an existing namespace in the schema, but Fictional_field is our own field inside that namespace.