Parsing Data
Security Requirements and Controls
Change parserspermission
When you send logs and metrics to LogScale for ingestion, they need to be parsed before they are stored in a repository. This is the case for all input channels except LogScale's structured ingest endpoint which stores data as-is.
A parser takes text as input; the text can be structured text (like JSON) or unstructured text (like syslog or application stdout). It then extracts fields which are stored along with the original text in the repository.
Figure 41. Flow
Choosing a Parser
A client sending data to LogScale must specify which repository to store the data in and which parser to use for ingesting the data. There are multiple ways to do so, depending on your preferences and clients, but if you are in doubt, using an Ingest Token and assigning a parser to the token is a good way to go, as it allows you to change parsers from within LogScale, without modifying outside clients.
Pre-built Parser from the Marketplace
LogScale has a Marketplace, which you can access inside your repositories and views. The Marketplace has a number of pre-built packages, many of which include parsers. These parsers normalize the ingested data to follow the CrowdStrike Parsing Standard (CPS) 1.0, which makes it easy to ingest multiple data sources, and search across them.
Built-In Parsers
LogScale ships with some parsers for common log formats through the Built-in Parsers. These parsers can parse formats such as JSON and accesslog, but they serve best as examples for writing your own parsers, as they cannot be extended and have limited functionality.
Creating a Custom Parser
Writing a custom parser allows you to take full control of which fields are extracted during ingest and which datasources events are stored in.
Parsers are written in the CrowdStrike Query Language (the same you use for searching). Read on in Creating a Parser to learn more.