Monitoring Alert Execution through the humio-activity Repository
The humio/activity package provides a wealth of information about activity within LogScale and should be installed to help monitor alerts.
The execution and any errors generated when executing alerts can be tracked in the humio-activity repo by examining the category field:
Alert mark standard alerts
The subCategory then indicates whether the event relates to the execution of the Alert, Query or Action.
Checking the severity field will indicate the level of the event:
Infoentries are used to indicate when an alert has been triggered
Warningindicates an issue either with the alert, reading the result, or triggering actions, or where an alert has not been triggered due to throttling
Errorindicates an error, for example running the query or trigger
The following additional fields in each event contain more detailed information for each alert invocation or error; for a full example event, see Alert Raw Event Example:
|ID of the triggered action; only set for the invocation of a specific action
|List of action ids for when an alert trigger has been triggered
|Unique id for the invocation of an action, can be used to correlate logs, same commenas for actionId
|List of action invocation ids for when an alert has been triggered
|Name of the action that generated an error or would have been triggered
|Name of the triggered action; only set for the invocation of a specific action
|ID of the alert
|Name of the alert
|The timestamp when the alert was triggered.
|Name of the repository or view
|The eventId when an alert trigger on an event
|The number of the events returned by the query; by default all queries return a maximum of 200. Where no events were returned by the query the value will be 0.
|A detailed error message that will include errors at the cluster-level that may have contributed; for example permission, API, or network issues
|The external id of the running query
|The timestamp of the last time the alert triggered
|The error or warning message for the alert
|The alert query executed
|The number of events processed to return the final result set.
|Time taken in milliseconds to execute the query. This value can be used to help indicate the load of the query (and therefore any optimization or refinement), or to find outliers during execution.
|A guide to the warning or error and how to resolve or identify more information
|The user the query runs on behalf of (run-as-user)
|ID of the view for the alert