Aggregation expressions are always function calls. These functions can combine their input into a new structure or emit new events into the output stream.

A query becomes an aggregation query if it uses at least one aggregate function like sum(), count() or avg().

For example, the query count() takes a stream of events as its input, and produces a single record containing a _count field.

Below are some examples:

loglevel = ERROR 
| timechart()

x := y * 2 
| bucket(function=sum(x))

Live queries are executed by LogScale during ingestion and are used to provide 'live' information from the ingested data. Because the data is processed when it is ingested, the results can be returned quicker than if the system waited for the data to be ingested and stored on disk. This is particularly useful in dashboards, but also for built-in alerting system.

When executing a live query, the time interval is always a time window relative to now, for example the last 5 minutes or the last day. Live queries are made up of the historical element and the live element, with the live element directly querying data that is being actively ingested.

Live queries are always being executed if they are being actively or regularly polled for results. This means that results are immediately available when opening a dashboard that is using live queries. If live queries are not polled the server will stop them at some point. The following describes when live queries are stopped:

The UI will try to stop live queries, when they are not used anymore. When submitting a new search, the previous one will be stopped.

No matter what kind of query you write, CQL provides a range of different tools and components designed to provide the results you need:

For more information, refer to:

One of the most basic queries in LogScale is to search for a string in one or more fields of events, in which all fields with the exception of a few are searched, including @rawstring.

Fields that are the exception include:

Free text filters enable searching the data, including the original raw string, without having to specify a field or more specific or defined value. This can be a simpler, and faster, method of finding and filtering data before extracting more specific content. It does not account for fields added or removed within the pipeline. Furthermore, free-text searches are only supported before an aggregate query function- after the initial aggregate function, a free-text search then refers to any text filter not specific to a field.

Conversely, a free-text search that is applied to a parser differs. Instead, the event is processed where the free-text search occurs. For this, using Field Filters within a parser avoids more general and therefore less relevant matches.

Free-text search does not specify what order in which fields are searched, because the order fields are checked only matters when fields are being extracted. Any match will let the event pass the filter. However, when extracting fields using a regular expression, matches can yield extracted fields that are at times unpredictable. For this reason, LogScale recommends user's try to match data on @rawstring when extracting fields.

Field filters allow users to query the values of specific event fields, both in text and numerical form. This is useful when data has been parsed and a specific value has been extracted, then assigned to a field. With field filters, users are able to distinguish between different values- for example, an original log line might contain multiple IP addresses, but the user wants to extract information based specifically on the IP address of the remote service associated with the organization. Querying specific fields, especially when combined with standardized parsing using the Pa-Sta standard, provides users with a focused and specific result.

For this filter, if "x = y" is the basic format for an expression, 'x' will always be a field name, and 'y' will be one of the following:

Regular expression filters operate either through a simple regular expression declaration or by using the regex() function. Different regex syntax operations are significantly different in performance: the /regex/ searching over all fields is slower compared to using either regex() or /regex/ that search on a single, specific field instead.

For more information, see:

LogScale correlates information by establishing relationships between two event data sets, known as the Primary Query and the Subquery, using joins. Joins match events from the two data sets, and are typically described according to how the combination of the two sets is attained.

These data sets can come from a LogScale repository, a search result, or a previously uploaded flat file.

This data combination is essential for various analytical needs, such as:

There are three different types of joins:

Lookup files provide extra context to event data by allowing you to add, replace, or filter fields in search results. To use them, you must upload a CSV or JSON file to a repository or view, and the dedicated Lookup files page assists in managing these files.

Once uploaded, these files are synchronized across all nodes in the cluster, though a brief delay may occur before updates are fully available for querying. The available operations for managing these files include creation, uploading, updating, exporting, and asset sharing.