This query function may be used to select events in which the given field contains particular values. For instance, you might want to monitor events in which log messages contain error, warning, or other similar words in log entries, or perhaps particular numeric values in other fields.

Although this query function allows for only three parameters, it is very useful and versatile. For the first parameter, you would specify the field on which to filter data. The second parameter sets whether the search should be case-insensitive. The third parameter would be the string or multiple strings on which to match the contents of the field.

ParameterTypeRequiredDefaultDescription
field[a]stringrequired  The field on which to filter events.
ignoreCasestringoptional[b]false Allows for case-insensitive searching.
valuesArray of stringsrequired  The values on which to match the field. Only one match is required. Values can contain wildcards (i.e., *).

[a] The argument name field can be omitted.

[b] Optional parameters use their default value unless explicitly set

Hide omitted argument names for this function

Show omitted argument names for this function

Hide negatable operation for this function

Show negatable operation for this function

in() Examples

Suppose you have a repository which is ingesting data from a few web servers. And suppose that you want to get a list of events in which the user received the HTTP code 404, for web pages Not Found. You could do that easily with this query:

logscale
status = 404

As this suggests, the field on which to check is status.

Suppose further that you want to get a list of events in which the user received the HTTP codes 422 and 200. Those codes represent respectively Unable to be processed, and Successful. You could get those events with the in() function like so:

logscale
in(status, values=["422","200"])

Using the status field for the first parameter; for the second parameter, the two statuses are listed, separated by commas, within an array — within square-brackets. Incidentally, if you wanted to include string values instead of numbers, each string value would have to be contained within double-quotes.

The screenshot in Figure 109, “in() Example” below shows how this would look in the LogScale interface.

in() Example

Figure 109. in() Example


There are a few other HTTP codes related to errors besides these two. You could list all of them in the array, or you could add the wildcard (i.e., *) like this:

logscale
in(status, values=["4*"])

This will return all events in which the status has a value starting with 4. Notice that even though only one value is given, you have to include the square-brackets. Also, notice that since the wildcard is used, the double-quotes is required.

Using the field parameter in addition to the =~ syntax:

logscale
in(field=loglevel, values=["ERROR", "WARN"])

Negating an in() filters:

logscale
!in(field=loglevel, values=["ERROR", "WARN"])

and

logscale
loglevel =~ !in(values=["ERROR", "WARN"])

Categorize events based on values in more fields

Query
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] result{{Result Set}} repo --> 0 0 --> result
logscale
case {     in(srcIP, values=["192.168.1.*"]) 
        | type := "Internal";     !in(loglevel, values=["DEBUG", "INFO"]) 
        | type := "Critical"; | type := "Other" }
Introduction

The in() function can be used to select events in which the given field contains specific values. It is possible to combine the in() with a case statement to categorize events. In this more advanced example, a case statement is used to categorize events based on the fields srcIP and loglevel, using both in() and negated in(). Notice that the semi-colon is used to end the different logical expressions.

Example incoming data might look like this: 

srcIP=192.168.1.5 loglevel=ERROR status=404 user=admin
    srcIP=10.0.0.1 loglevel=INFO status=200 user=user1
    srcIP=172.16.0.5 loglevel=WARN status=422 user=user2
    srcIP=192.168.1.15 loglevel=ERROR status=500 user=admin
    srcIP=10.0.0.12 loglevel=DEBUG status=302 user=user1

Step-by-Step

  1. Starting with the source repository events

  2. flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] result{{Result Set}} repo --> 0 0 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;

    Returns all events with values starting with 192.168.1.* followed by anything in the scrIP field and then creates a new field named type with the assigned value Internal for the returned results. Notice that since the wildcard is used, the double-quotes is required. Next, the query searches for events where the field loglevel does not contain the values DEBUG or INFO and assigns the value Critical to the returned results in the type field. For anything else, it sets the value in the type field to Other. In this example, INFO and DEBUG will therefore be set to Other. The above case statement can also be expressed like this: If the sourceIP equals the value 192.168.1.* followed by anything, then identify the type field as Internal. If it is not equal to the loglevel of debug or info, then identify the type field as Critical. If it does not match either of the above, identyfy the type field as Other.

    logscale
    case {     in(srcIP, values=["192.168.1.*"]) 
        | type := "Internal";     !in(loglevel, values=["DEBUG", "INFO"]) 
        | type := "Critical"; | type := "Other" }

  3. Event Result set

Summary and Results

The query is used to to categorize events and define their type.

Sample output from the incoming example data:

srcIPloglevelstatususertype
192.168.1.5ERROR404adminInternal
10.0.0.1INFO200user1Other
172.16.0.5WARN422user2Critical
192.168.1.15ERROR500adminInternal
10.0.0.12DEBUG302user1Other

Perform case-insensitive match on field

Query
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] result{{Result Set}} repo --> 0 0 --> result
logscale
in(loglevel, ignoreCase=true, values=["error", "warn"])
Introduction

The in() function can be used to select events in which the given field contains specific values. It is possible to perform case-insensitive searches on a field using the in() function. In this example, the loglevel field is searched for occurrences of either error or warning.

Example incoming data might look like this: 

srcIP=192.168.1.5 loglevel=ERROR status=404 user=admin
    srcIP=10.0.0.1 loglevel=INFO status=200 user=user1
    srcIP=172.16.0.5 loglevel=WARN status=422 user=user2
    srcIP=192.168.1.15 loglevel=ERROR status=500 user=admin
    srcIP=10.0.0.12 loglevel=DEBUG status=302 user=user1

Step-by-Step

  1. Starting with the source repository events

  2. flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] result{{Result Set}} repo --> 0 0 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;

    Returns all events in which the loglevel field has the value error or warning. As it is case-insensitive, it returns all occurences of the specified values in all their variants, regardless of the case.

    logscale
    in(loglevel, ignoreCase=true, values=["error", "warn"])

  3. Event Result set

Summary and Results

The query is used to perform case-insensitive searches on a specific value in a given field. This is useful when searching for strings where values may appear in both both upper and lower case to ensure that all events are extracted.

Sample output from the incoming example data:

srcIPloglevelstatususer
192.168.1.5ERROR404admin
172.16.0.5WARN422user2
192.168.1.15ERROR500admin

Search status field for all status codes starting with "1" or "2"

Query
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] result{{Result Set}} repo --> 0 0 --> result
logscale
in(status, values=["1*", "2*"])
Introduction

The in() function can be used to select events in which the given field contains specific values. It is possible to use wildcards with the in() function to select for example all status codes starting with "1" or "2". Notice that "" must be used around the *.

Example incoming data might look like this: 

srcIP=192.168.1.5 loglevel=ERROR status=404 user=admin
    srcIP=10.0.0.1 loglevel=INFO status=200 user=user1
    srcIP=172.16.0.5 loglevel=WARN status=422 user=user2
    srcIP=192.168.1.15 loglevel=ERROR status=500 user=admin
    srcIP=10.0.0.12 loglevel=DEBUG status=302 user=user1

Step-by-Step

  1. Starting with the source repository events

  2. flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] result{{Result Set}} repo --> 0 0 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;

    Returns all events in which the status has a value starting with either 1 or 2. Notice that since the wildcard is used, the double-quotes is required.

    logscale
    in(status, values=["1*", "2*"])

  3. Event Result set

Summary and Results

The query is used to search status field for status codes starting with a given integer.

Sample output from the incoming example data:

srcIPloglevelstatususer
10.0.0.1INFO200user1

Search two fields for multiple values in either first field or second field

Query
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] result{{Result Set}} repo --> 0 0 --> result
logscale
case
        { in(srcIP, values=["10.1.168.2", "127.0.0.1"]); 
        in(targetIP, values=["10.0.0.1", "192.168.1.12"]); }
Introduction

The in() function can be used to select events in which the given field contains specific values. Sometimes it may be necessary to search for multiple values in two different fields in the same query string. Though the in() function cannot directly be combined with an OR clause, it is possible to use the in() function in a case statement to produce the same output as an OR. In this example, the query will look for events in either the srcIP field or the targetIP.

Step-by-Step

  1. Starting with the source repository events

  2. flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] result{{Result Set}} repo --> 0 0 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;

    Filters for events in the srcIP field that contains the values 10.1.168.2 or 127.0.0.1 and filters for events in the targetIP field that contains the values 10.0.0.1 or 192.168.1.12. The returned results would be events from both fields. Notice that because it is a case statement, it executes and returns whether either field contains the corresponding values in the array.

    logscale
    case
        { in(srcIP, values=["10.1.168.2", "127.0.0.1"]); 
        in(targetIP, values=["10.0.0.1", "192.168.1.12"]); }

  3. Event Result set

Summary and Results

The query is used to query two fields for multiple/specific values in either first field or second field.