Queries and Querying

Queries and querying are the cornerstone of unlocking data's potential for insight. The query language offered to users for designing queries is the CrowdStrike Query Language (CQL), which allows users to design complex queries that accomplishes a wide range of analysis and visualization.

The Basics

In general, there are a few rules that will help users who are learning CQL and beginning to design queries:

  • Specificity matters. The more specific the query, the faster the query will run and the more focused the results will be

  • Adhering to an ordered flow when writing queries will help with CPU costs, memory usage, and increase efficiency

  • Filter first, then perform data formatting and modification

CrowdStrike Query Language (CQL)

CrowdStrike Query Language (CQL) is the query syntax to use when composing queries to retrieve, process and analyze data in Falcon LogScale. To derive the best possible result, queries must be written using the appropriate syntax.

CQL Components

  • Query Filters: Filters reduce query results to only relevant data by using free-text filters to grep data, filtering based on fields, stipulating acceptable field values, or using regular expressions for matching field contents.

  • Operators: Several operators exist for filtering- logical operators and comparison operators narrow search results to only what's most important.

  • Adding Fields: Creating new fields when querying data improves result sets and gives the ability to construct more complex queries.

  • Conditional Statements: CQL doesn't provide a typical conditional syntax, but there are ways to evaluate data on a conditional basis by using a case or a match statement.

  • Query Joins: Queries can be used to filter or enrich other queries to obtain a combined result using the join() function.

Filtering and Analysis

LogScale query filters enable powerful search capabilities through free text matching, field-specific filters, and regular expressions, with each type offering distinct ways to find and filter event data.

Free Text Filters

One of the most basic queries in LogScale is to search for a string in one or more fields of events, in which all fields with the exception of a few are searched, including @rawstring.

Fields that are the exception include:

  • @id

  • @timestamp

  • @ingesttimestamp

  • Tag fields

Field Filters

Field filters allow users to query the values of specific event fields, both in text and numerical form. This is useful when data has been parsed and a specific value has been extracted, then assigned to a field.

For this filter, if "x = y" is the basic format for an expression, 'x' will always be a field name, and 'y' will be one of the following:

  • A literal entry, i.e. 3.14, "Sumatra", ok

  • A wildcard pattern allows users to search for values matching the beginning or end of a value or value. An asterisk denotes the area currently in question:

    • deli* (delicious, delilah, delimiter)

    • mega* (megatron, megan)

  • A regular expression, supporting many of the standard regular expression semantics

Correlation

Correlation is the ability to look at multiple relationships within the data and then make a decision or match based on that combination.

Joins

LogScale correlates information by establishing relationships between two event data sets, known as the Primary Query and the Subquery, using joins. Joins match events from the two data sets, and are typically described according to how the combination of the two sets is attained.

These data sets can come from:

  • A LogScale repository

  • A search result

  • A previously uploaded flat file

There are three different types of joins:

  • Left joins

  • Inner joins

  • Right joins

Lookup Files

Lookup files provide extra context to event data by allowing you to add, replace, or filter fields in search results. To use them, you must upload a CSV or JSON file to a repository or view, and the dedicated Lookup files page assists in managing these files.

Lookup files using CSV format

When using CSV for lookup files, the following rules apply:

  • Individual fields should be separated by a comma (,)

  • Whitespace is always included in the imported fields, the input takes the literal contents split by the comma character

  • Fields can optionally be quoted by double quotes, for example to include commas in the imported values

  • The first line of the CSV is interpreted as the column header

Lookup files using JSON format

When using JSON files, two different formats are supported:

  • Object-based

  • Array-based