Available:slidingTimeWindow() v1.174.0

The slidingTimeWindow() function is available from version 1.174.0.

The slidingTimeWindow() function applies an aggregation to a moving time-based window of events in a sequence. It is useful for calculating metrics over a fixed time period, allowing for time-based trend analysis and data smoothing.

The difference between slidingTimeWindow() and window() is that window() spans multiple buckets and accumulates events inside the bucket, whereas slidingTimeWindow() does not use buckets, but simply accumulates across the incoming events within a specified span.

For more information about sequence functions and combined usage, see Sequence Query Functions.

ParameterTypeRequiredDefault ValueDescription
currentenumoptional[a] include Controls whether to include the current event in the window calculation.
   Values
   excludeExclude current event in window calculation
   includeInclude current event in window calculation
function[b]array of aggregate functionsrequired   The aggregator function(s) to apply to each time window. It only accepts functions that output at most a single event.
spanstringrequired   The duration of the time window (for example, 1h, 30m, 1d).
timestampfieldstringoptional[a] Either @timestamp or @ingestTimestamp depending on what is selected for the query. Specifies the field to use as the timestamp for calculations.

[a] Optional parameters use their default value unless explicitly set.

[b] The parameter name function can be omitted.

Hide omitted argument names for this function

Show omitted argument names for this function

Note

  • The slidingTimeWindow() function must be used after an aggregator function (for example, head(), sort(), bucket(), groupBy() timeChart()) to ensure event ordering, as the slidingTimeWindow() function requires a specific order to calculate cumulative values correctly.

  • Only functions (for example, sum(), avg(), count()) that output a single event can be used in the sub-aggregation because the slidingTimeWindow() function needs a single value to add to its running total for each event.

  • The window can contain a maximum of 10000 events.

  • Events must be sorted in order by timestamp. Unordered or missing timestamps will result in errors.

Click + next to an example below to get the full details.

Detect Event A Happening X Times Before Event B Within a Specific Timespan

Detect event A happening X times before event B within a specific timespan using the slidingTimeWindow() function combined with groupBy()

Query
logscale
head()
| groupBy(
    key,
    function=slidingTimeWindow(
        [{status="failure" | count(as=failures)}, selectLast(status)],
        span=3s
    )
  )
| failures >= 3
| status = "success"
Introduction

In this example, the slidingTimeWindow() function is used with the groupBy() function to detect event A happening X times before event B within a specific timespan.

The query will detect instances where there are 3 or more failed attempts followed by a successful attempt, all occurring within a 3-second window.

Note that the slidingTimeWindow() function must be used after an aggregator function to ensure event ordering. Also note that the events must be sorted in order by timestamp to prevent errors when running the query. It is possible to select any field to use as a timestamp.

Example incoming data might look like this:

@timestampkeystatus
1451606300200cfailure
1451606300400cfailure
1451606300600cfailure
1451606301000afailure
1451606302000afailure
1451606302200afailure
1451606302300afailure
1451606302400bfailure
1451606302500afailure
1451606302600asuccess
1451606303200bfailure
1451606303300csuccess
1451606303400bfailure
1451606304500afailure
1451606304600afailure
1451606304700afailure
1451606304800asuccess
Step-by-Step

  1. Starting with the source repository events.

  2. logscale
    head()

    Selects the oldest events ordered by time.

  3. logscale
    | groupBy(
    key,
    function=slidingTimeWindow(
        [{status="failure" | count(as=failures)}, selectLast(status)],
        span=3s
    )
  )

    Groups the events by a specified key (for example, a user ID or IP address), then creates a sliding time window of 3 seconds (with a span of 3 seconds).

    Furthermore, it filters all the failed attempts where the field status contains the value failure, makes a count of all the failed attempts, and returns the results in a field named failures, calculates the timespan of the failures, retrieves the timestamp of the last failure, and selects the status of the last event.

  4. logscale
    | failures >= 3

    Filters for windows with 3 or more failures.

  5. logscale
    | status = "success"

    Filters for partitions containing the value success in the status field.

  6. Event Result set.

Summary and Results

The query is used to detect event A happening X times before event B within a specific timespan. It looks for instances where there were 3 or more failed attempts followed by a successful attempt, all occurring within a 3-second window. Using a sliding time window of 3 seconds, provides a more precise time constraint compared to the usage of partition() in Detect Event A Happening X Times Before Event B.

The query can be used to detect potential brute force attack patterns within a specific timeframe. Note that the effectiveness of this query depends on the nature of your data and the typical patterns in your system.

Sample output from the incoming example data:

keyfailuresstatus
a5success
a7success

Detect Two Events Occurring in Quick Succession

Detect event B occurring quickly after event A using the slidingTimeWindow() function

Query
logscale
head()
| slidingTimeWindow(
    [{event = "A" | count(event, as=countAs)}, selectLast(event)], 
    span=1s
  )
| countAs > 0
| event = "B"
Introduction

In this example, the slidingTimeWindow() function is used to detect event B occurring quickly after event A.

Note that the slidingTimeWindow() function must be used after an aggregator function to ensure event ordering. Also note that the events must be sorted in order by timestamp to prevent errors when running the query. It is possible to select any field to use as a timestamp.

Example incoming data might look like this:

@timestampevent
1451606300500A
1451606301000B
1451606302000A
1451606304000B
Step-by-Step

  1. Starting with the source repository events.

  2. logscale
    head()

    Selects the oldest events ordered by time.

  3. logscale
    | slidingTimeWindow(
    [{event = "A" | count(event, as=countAs)}, selectLast(event)], 
    span=1s
  )

    Creates a sliding time window of 1 second. Within each window it counts the occurrences of event A, returning the results in a new field named countAs, and selects the event type of the last event in the window.

  4. logscale
    | countAs > 0

    Filters for windows where at least one event A occurred.

  5. logscale
    | event = "B"

    Checks if the last event in the window is event B.

  6. Event Result set.

Summary and Results

The query is used to detect instances where event B occurs quickly (within 1 second) after event A. The span parameter configures the interval, allowing this to be customized.

Sample output from the incoming example data:

countAsevent@timestamp
1B1451606301000

The query is useful for identifying sequences of events that happen in quick succession, which could indicate specific patterns of behavior or system interactions.