LogScale Query Language defined

LogScale Query Language (LQL) is the query syntax to use when composing queries to retrieve, process and analyze data in Falcon LogScale. Built around a chain of data-processing commands linked together, each expression passes its result to the next expression in the sequence, allowing you to create complex queries by combining expressions. This architecture is similar to command pipes in Unix and Linux shells.

Events ingested and parsed in LogScale can be any type of text based data, can be both structured and unstructured, and can originate anywhere from application logs and infrastructure events to networks, security-related devices, or applications.

Much like a query for an SQL database, LQL queries are written to include or exclude values from a repository or view. However, unlike most SQL queries, you can also do calculations and transform data.

Transformation and Aggregation Queries

The two most common queries are transformation and aggregation queries.

Transformation queries (also called filter expressions) filter input or add/remove/modify fields on each event. These include filter expressions like:

logscale
time:dayOfWeekName(field=@timestamp, as=the_day)

This uses the time:dayOfWeekName() function to extract the day of the week from the timestamp for each event, format that value to return the name of the day of the week, then put that value in a field titled 'the day'. You can see the results in the screenshot in Figure 1.

A subset of the available query functions are known as Transformation Functions, for example regex(), in() or eval(). Just like the examples above, they only add/remove/modify fields and never produce new (additional) events as output.

If a query consists solely of transformation expressions it is known as a filter query or a transformation query. This kind of query is required when connecting Views with repositories.

Aggregation queries are always function calls. These functions combine their input into a new structure, or emit new events into the output stream.

Queries become an aggregation query if they use at least one aggregate function, like:

For example, the query count() takes a stream of events as its input, and produces a single record containing a _count field.

Below are some examples:

logscale
loglevel = ERROR | timechart()
logscale
x := y * 2 | bucket(function=sum(x))

LogScale Query Language Syntax

No matter what kind of query you write, LQL provides a range of different tools and components designed to provide the results you need:

  • Query Filters

    Filters reduce query results to only relevant data by using free-text filters to grep data, filtering based on fields, stipulating acceptable field values, or using regular expressions for matching field contents. See Query Filters.

  • Operators

    Several operators exist for filtering- logical operators and comparison operators narrow search results to only what's most important. See Operators.

  • Adding Fields

    Creating new fields when querying data improves result sets and gives the ability to construct more complex queries. To do this, use the := syntax, the as syntax, a regex, or eval() function. See Adding Fields.

  • Conditional Statements

    LQL doesn't provide a typical conditional syntax, but there are ways to evaluate data on a conditional basis by using a case or a match statement. See Conditional Evaluation.

  • Query Joins

    Queries can be used to filter or enrich other queries to obtain a combined result using the join() function. One query can be used to filter or enrich another, thus obtaining a combined result. See join() Syntax.

  • Array Syntax

    Array functions allow you to extract, create and manipulate items embedded in arrays, or to interpret arrays, within events using a syntax similar to JSON.

  • Function Syntax

    LogScale provides built-in query functions to obtain values or reduce results. You can also combine them to create your own. See Query Functions.

  • Time Related Syntax

    Time syntax defines how to convert or translate timestamps and other time-related values, including specifying and calculating time in relation to other times. Rate Unit Conversion is also important. See Relative Time Syntax.

  • Regular Expression Syntax

    Regular expressions in LQL are similar to many other regular expression environments, with some notable differences. See Regular Expression Syntax.

For a more in-depth exploration of LogScale's query language, visit our documentation here: Query Language Syntax