Detect All Occurrences of Event A Before Event B

Detect all occurrences of event A before event B (brute force attack) using the partition() function combined with groupBy()

Query

logscale

head()
| groupBy(
      key,
      function = partition(
          condition=test(status=="success"),
          split="after",
          [
              { status="failure" | count(as=failures) }, 
              range(@timestamp, as=timespan), 
              selectLast(status)
          ]
      )
  )
| failures >= 3
| status = "success"

Introduction

In this example, the partition() function is used with the groupBy() function to detect all occurrences of event A before event B (brute force attack).

The query will detect instances where there were 3 or more failed attempts followed by a successful attempt within the specified 10-second window.

Note that the partition() function must be used after an aggregator function to ensure event ordering. Also note that the events must be sorted in order by timestamp to prevent errors when running the query. It is possible to select any field to use as a timestamp.

Example incoming data might look like this:

@timestamp	key	status
1451606300200	c	failure
1451606300400	c	failure
1451606300600	c	failure
1451606301000	a	failure
1451606302000	a	failure
1451606302200	a	failure
1451606302300	a	failure
1451606302400	b	failure
1451606302500	a	failure
1451606302600	a	success
1451606303200	b	failure
1451606303300	c	success
1451606303400	b	failure
1451606304500	a	failure
1451606304600	a	failure
1451606304700	a	failure
1451606304800	a	success

Step-by-Step

Starting with the source repository events.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0{{Aggregate}} 1{{Aggregate}} 2[/Filter/] 3{{Aggregate}} result{{Result Set}} repo --> 0 0 --> 1 1 --> 2 2 --> 3 3 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
head()
```
Selects the oldest events ordered by time.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0{{Aggregate}} 1{{Aggregate}} 2[/Filter/] 3{{Aggregate}} result{{Result Set}} repo --> 0 0 --> 1 1 --> 2 2 --> 3 3 --> result style 1 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
| groupBy(
      key,
      function = partition(
          condition=test(status=="success"),
          split="after",
          [
              { status="failure" | count(as=failures) }, 
              range(@timestamp, as=timespan), 
              selectLast(status)
          ]
      )
  )
```
Groups the events by a specified key (for example, a user ID or IP address), filters for successful events (filters for events that meet the defined condition for the field status that must contain the value success), then splits the data after each successful event. Notice how the condition is provided as a non-aggregate subquery.
Furthermore, it filters all the failed attempts where the field status contains the value failure.
Makes a count of all the failed attempts, and returns the results in a field named failures, calculates the timespan of the failures, and selects the status of the last event. Calculating the timespan of the failure sequence, is useful for analysis.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0{{Aggregate}} 1{{Aggregate}} 2[/Filter/] 3{{Aggregate}} result{{Result Set}} repo --> 0 0 --> 1 1 --> 2 2 --> 3 3 --> result style 2 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
| failures >= 3
```
Filters for partitions with 3 or more failures.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0{{Aggregate}} 1{{Aggregate}} 2[/Filter/] 3{{Aggregate}} result{{Result Set}} repo --> 0 0 --> 1 1 --> 2 2 --> 3 3 --> result style 3 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
| status = "success"
```
Filters for partitions containing the value success in the status field.
Event Result set.

Summary and Results

The query is used to detect all occurrences of potential brute force attack patterns. It looks for instances where there were 3 or more failed attempts (event A) followed by a successful attempt (event B), regardless of the time between failures. The timespan between each attempt is reported, which could be used to identify brute force attacks.

Sample output from the incoming example data: