Detect Continuously Upwards Going Trend

Detect continuously upwards going trend using the slidingWindow() function combined with neighbor()

Query

logscale
head()
| neighbor(value, prefix=prev)
| change := value - prev.value
| slidingWindow(
    [
         {change >= 0 | count(as=positiveTrend)},
         {change < 0  | count(as=negativeTrend)}
    ],
    events=2
    )
| positiveTrend >= 2

Introduction

The slidingWindow() function can be used to detect continuously upwards going trend.

In this example, the slidingWindow() function combined with neighbor() is used to detect continuously upwards going trend. It looks for sequences where the value is consistently increasing or staying the same over at least two consecutive measurements.

Note that sequence functions must be used after an aggregator function to ensure event ordering.

Example incoming data might look like this:

value
3
5.5
4
6
10

Step-by-Step

  1. Starting with the source repository events.

  2. logscale
    head()

    Selects the oldest events ordered by time.

  3. logscale
    | neighbor(value, prefix=prev)

    Creates a new field named prev.value containing the value from the previous event.

  4. logscale
    | change := value - prev.value

    Calculates the change between the current value and the previous value, and assigns the returned results to a field named change.

  5. logscale
    | slidingWindow(
    [
         {change >= 0 | count(as=positiveTrend)},
         {change < 0  | count(as=negativeTrend)}
    ],
    events=2
    )

    Creates a sliding window of 2 events. Within each window, it counts changes equal to zero or higher (positive or zero changes) and returns the results in a field named positiveTrend, and then also counts the negative changes and returns the results in a field named negativeTrend.

  6. logscale
    | positiveTrend >= 2

    Filters for windows where there are at least 2 positive (or zero) changes.

  7. Event Result set.

Summary and Results

The query is used to detect a continuous upwards trend in a series of values. The query can be used to monitor system metrics for consistent increases (for example, memory usage, CPU load) and to identify potential anomalies in time-series data.

Sample output from the incoming example data:

valuepositiveTrendnegativeTrendchangeprev.value
102046