Filter For Items Not Part of Data Set Using !match() | LogScale Query Examples

Filter For Items Not Part of Data Set Using `!match()`

Find the set difference using the match() function with negation

Query

logscale

src_ip=*
| !match("known_ips.csv", field=src_ip)

Introduction

The match() function can be used with a negation to filter for items that are not part of a data set.

In this example, the match() function is used with a negation to search and find IP addresses, that are not part of a known list known_ips.csv.

Example incoming data might look like this:

timestamp	src_ip	dst_ip	src_port	dst_port	protocol	bytes_sent	bytes_received
2025-04-01T07:00:00Z	192.168.1.101	10.0.0.50	52431	443	TCP	1024	2048
2025-04-01T07:00:01Z	172.16.0.24	8.8.8.8	33221	53	UDP	64	512
2025-04-01T07:00:02Z	192.168.1.150	172.16.0.100	49223	80	TCP	2048	4096
2025-04-01T07:00:03Z	10.0.0.75	192.168.1.1	55678	22	TCP	512	1024
2025-04-01T07:00:04Z	192.168.1.200	1.1.1.1	44556	53	UDP	64	512

Step-by-Step

Starting with the source repository events.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0["Expression"] 1["Expression"] result{{Result Set}} repo --> 0 0 --> 1 1 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
src_ip=*
```
Filters for all events that have a src_ip field.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0["Expression"] 1["Expression"] result{{Result Set}} repo --> 0 0 --> 1 1 --> result style 1 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
| !match("known_ips.csv", field=src_ip)
```
Excludes (filters out) any events where the src_ip field matches entries in the file known_ips.csv, and returns a list of IP addresses that are not found in the specified file. The negation operator is used to return non-matching results.
Event Result set.

Summary and Results

The query is used to search for unknown or unexpected source IP addresses matched up againt a known list. This is useful for detecting potential security theats and monitoring for unauthorized network access.

Sample output from the incoming example data: