Automations

Automations and their use allows users to automate tasks within LogScale, including queries and notifications, reducing a user's "hands on" time and providing consistency. They can also provide notifications using customizable actions when events occur. This is accomplished using triggers and alerts. There are several trigger options:

  • Triggers

  • Actions

Triggers are queries that run either continuously or on a schedule depending on how they are configured, and allow for alerts and notifications. Actions can arrive through various means, including email and Slack, to inform users regarding events transpiring that users have set their instance of LogScale to identify.

Triggers

Triggers act as a way for users to produce certain actions when a query generates a result- instead of routinely checking LogScale and executing queries manually or programmatically, triggers detect problems real-time, as they occur. There are four types of triggers:

  • Scheduled searches

  • Aggregate alerts

  • Filter alerts

  • Legacy alerts

Scheduled searches and alerts are both mechanisms for triggering actions based on query results, but they differ in how and when they execute.

  • Scheduled searches use static queries that run on a regular schedule against stored data.

  • Alerts use continuous, live queries on data as it is ingested.

In both cases, when a query returns a result, one or more actions are set off. The use of triggers facilitates automation, such as notifying analysts and administrators via various integrations, including email or forwarding results to another repository.

Scheduled Searches

Scheduled searches are static queries that run on a schedule. At a set interval, the query runs, and any result triggers one or more actions that are set by the user. This is particularly useful not only from a system monitoring and maintenance standpoint, but also from a reporting standpoint- if there are organizations who wish to remain updated on specific aspects of the system on a daily or weekly basis, that can be automated using scheduled searches. If preferred, users who wish to have their results in dashboard format can use the Schedule PDF Reports functionality.

For more information, see the documentation here: Scheduled searches

Aggregate Alerts

Aggregate alerts are based on aggregating queries within a search interval and can be run concurrently, in such a way that no time interval is overlooked or checked twice. If any problem occurs systemically, the query is re-run to completion to guarantee that alerts can catch up if the system has been down. This behavior guarantees reliability of the alert in case of an infrastructure failure.

For more information, see the documentation here: Aggregate alerts

Filter Alerts

Filter alerts are based on non-aggregate queries, and are designed to trigger an action when a live query filters an event. Each event elicits the action(s) associated with the trigger, and evaluates each event individually as it enters LogScale. Filter alerts will process events up to and including catching up for past events, for up to 24 hours.

For more information, see the documentation here: Filter alerts

Legacy Alerts

Legacy alerts, formerly known as Standard Alerts, are triggered by queries that generate a result set- they work by returning results from an aggregate query, and when the query returns a result with one or more rows, the alert is triggered. If not already an aggregate query, tail(200) is appended to the query to make it an aggregate query.Legacy alerts are not recommended, because aggregate queries particularly using join operations are not computed consistently across the two queries. The queries are executed against the data being ingested, which may have changed between executing each query. Because the datasets from the primary query and subquery are inconsistent, the results of the join are also inconsistent.

Actions

Actions are the user-defined, automated outcome of triggers, and notify users or administrators of an issue. These include options like sending an email or copying a matching event to a new repository for use in another dashboard.

Actions are executed in parallel, and is considered successful if any single action succeeds. When true, actions move to the next time window. Consequently, if one action fails but another succeeds, the overall automation is deemed successful, and the failed action will not be retried.

Action Types

Action types are the tools and their respective incident management platform that are capable of being used to provide notification of a potential problem to users and system managers.

There are several tools and incident management platforms that may be used to get notified of a potential problem or to bring a situation to someone's attention.

  • Email will send a styled email with the most important aspects of an alarm, including a hyperlink back to LogScale with the result. Email delivery services integrated with LogScale such as Postmark are supported, see Email Configuration for further details.

  • Falcon LogScale Repository sends events from a trigger to a LogScale repository. This can be used to summarize all such events, or to aggregate information from multiple triggers. The events from the trigger are parsed and ingested using the ingest token. If the ingest token has an associated parser, it is used; otherwise, the built-in-parser json-for-action is used.

  • OpsGenie is an alert routing and incident management system that can be used in conjunction with LogScale.

  • PagerDuty is an incident management platform that LogScale supports through the action system. LogScale provides a package that includes a template for a PagerDuty action, so users can customize elements of the actions.

  • Amazon S3 (S3) is an object storage service- the S3 action uploads query results from LogScale as files, and can be triggered by alerts or scheduled searches to export event data in either CSV or NDJSON format.

  • Slack Actions allows users to notify one or more channels in a Slack workspace.

  • The Lookup File action creates a CSV lookup file of the collected events, then uploads the file to LogScale in the requisite repository/view associated with the action. The resulting CSV lookup file can be used with match() functions and others.

  • VictorOps (Splunk On-Call) is a LogScale-provided package that provides a template for Splunk On-Call, so users can customize certain basic elements of the action.

  • Webhook actions can perform an HTTP(S) request to any URL and can therefore be used to integrate third-party services that LogScale doesn't have natively integrated.

For more information, see the documentation here: Automation Triggers Actions