This query function may be used to select events in which the given field contains particular values. For instance, you might want to monitor events in which log messages contain error, warning, or other similar words in log entries, or perhaps particular numeric values in other fields.
Although this query function allows for only three parameters, it is very useful and versatile. For the first parameter, you would specify the field on which to filter data. The second parameter sets whether the search should be case-insensitive. The third parameter would be the string or multiple strings on which to match the contents of the field.
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
field [a] | string | required | The field on which to filter events. | |
ignoreCase | string | optional[b] | false | Allows for case-insensitive searching. |
values | Array of strings | required | The values on which to match the field. Only one match is required. Values can contain wildcards (i.e., * ). | |
[b] Optional parameters use their default value unless explicitly set |
Hide omitted argument names for this function
Omitted Argument NamesThe argument name for
field
can be omitted; the following forms of this function are equivalent:logscalein("field",values=["value"])
and:
logscalein(field="field",values=["value"])
These examples show basic structure only.
Hide negatable operation for this function
Negatable Function OperationThis function is negatable, implying the inverse of the result. For example:
logscale!in()
Or:
logscalenot in()
For more information, see Negating the Result of Filter Functions.
in()
Examples
Suppose you have a repository which is ingesting data from a few web servers. And suppose that you want to get a list of events in which the user received the HTTP code 404, for web pages Not Found. You could do that easily with this query:
status = 404
As this suggests, the field on which to check is status.
Suppose further that you want to get a list of events in which the user
received the HTTP codes 422 and
200. Those codes represent respectively
Unable to be processed, and
Successful. You could get those events with the
in()
function like so:
in(status, values=["422","200"])
Using the status field for the first parameter; for the second parameter, the two statuses are listed, separated by commas, within an array — within square-brackets. Incidentally, if you wanted to include string values instead of numbers, each string value would have to be contained within double-quotes.
The screenshot in Figure 111, “in()
Example” below
shows how this would look in the LogScale interface.
Figure 111. in()
Example
There are a few other HTTP codes related to errors besides these two.
You could list all of them in the array, or you could add the wildcard
(i.e., *
) like this:
in(status, values=["4*"])
This will return all events in which the status has a value starting with 4. Notice that even though only one value is given, you have to include the square-brackets. Also, notice that since the wildcard is used, the double-quotes is required.
Using the field
parameter in addition to the
=~
syntax:
in(field=loglevel, values=["ERROR", "WARN"])
Negating an in()
filters:
!in(field=loglevel, values=["ERROR", "WARN"])
and
loglevel =~ !in(values=["ERROR", "WARN"])
Categorize events based on values in more fields
case { in(srcIP, values=["192.168.1.*"])
| type := "Internal"; !in(loglevel, values=["DEBUG", "INFO"])
| type := "Critical"; | type := "Other" }
The in()
function can be used to select
events in which the given field contains specific values. It is
possible to combine the in()
with a case
statement to categorize events. In this more advanced example, a
case statement is used to categorize events based on the fields
srcIP and loglevel,
using both in()
and negated
in()
. Notice that the semi-colon is used to
end the different logical expressions.
Example incoming data might look like this:
srcIP=192.168.1.5 loglevel=ERROR status=404 user=admin
srcIP=10.0.0.1 loglevel=INFO status=200 user=user1
srcIP=172.16.0.5 loglevel=WARN status=422 user=user2
srcIP=192.168.1.15 loglevel=ERROR status=500 user=admin
srcIP=10.0.0.12 loglevel=DEBUG status=302 user=user1
Step-by-Step
Starting with the source repository events
- flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] result{{Result Set}} repo --> 0 0 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;
Returns all events with values starting with
192.168.1.*
followed by anything in the scrIP field and then creates a new field named type with the assigned valueInternal
for the returned results. Notice that since the wildcard is used, the double-quotes is required. Next, the query searches for events where the field loglevel does not contain the valuesDEBUG
orINFO
and assigns the valueCritical
to the returned results in the type field. For anything else, it sets the value in the type field toOther
. In this example,INFO
andDEBUG
will therefore be set toOther
. The above case statement can also be expressed like this: If the sourceIP equals the value192.168.1.*
followed by anything, then identify the type field asInternal
. If it is not equal to the loglevel of debug or info, then identify the type field asCritical
. If it does not match either of the above, identyfy the type field asOther
.logscalecase { in(srcIP, values=["192.168.1.*"]) | type := "Internal"; !in(loglevel, values=["DEBUG", "INFO"]) | type := "Critical"; | type := "Other" }
Event Result set
The query is used to to categorize events and define their type.
Sample output from the incoming example data:
srcIP | loglevel | status | user | type |
---|---|---|---|---|
192.168.1.5 | ERROR | 404 | admin | Internal |
10.0.0.1 | INFO | 200 | user1 | Other |
172.16.0.5 | WARN | 422 | user2 | Critical |
192.168.1.15 | ERROR | 500 | admin | Internal |
10.0.0.12 | DEBUG | 302 | user1 | Other |
Perform case-insensitive match on field
in(loglevel, ignoreCase=true, values=["error", "warn"])
The in()
function can be used to select
events in which the given field contains specific values. It is
possible to perform case-insensitive searches on a field using the
in()
function. In this example, the
loglevel field is searched for occurrences of
either error
or warning
.
Example incoming data might look like this:
srcIP=192.168.1.5 loglevel=ERROR status=404 user=admin
srcIP=10.0.0.1 loglevel=INFO status=200 user=user1
srcIP=172.16.0.5 loglevel=WARN status=422 user=user2
srcIP=192.168.1.15 loglevel=ERROR status=500 user=admin
srcIP=10.0.0.12 loglevel=DEBUG status=302 user=user1
Step-by-Step
Starting with the source repository events
- flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] result{{Result Set}} repo --> 0 0 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;
Returns all events in which the loglevel field has the value
error
orwarning
. As it is case-insensitive, it returns all occurences of the specified values in all their variants, regardless of the case.logscalein(loglevel, ignoreCase=true, values=["error", "warn"])
Event Result set
The query is used to perform case-insensitive searches on a specific value in a given field. This is useful when searching for strings where values may appear in both both upper and lower case to ensure that all events are extracted.
Sample output from the incoming example data:
srcIP | loglevel | status | user |
---|---|---|---|
192.168.1.5 | ERROR | 404 | admin |
172.16.0.5 | WARN | 422 | user2 |
192.168.1.15 | ERROR | 500 | admin |
Search status field for all status codes starting with "1" or "2"
in(status, values=["1*", "2*"])
The in()
function can be used to select
events in which the given field contains specific values. It is
possible to use wildcards with the in()
function to select for example all status codes starting with "1"
or "2". Notice that ""
must be used around the
*
.
Example incoming data might look like this:
srcIP=192.168.1.5 loglevel=ERROR status=404 user=admin
srcIP=10.0.0.1 loglevel=INFO status=200 user=user1
srcIP=172.16.0.5 loglevel=WARN status=422 user=user2
srcIP=192.168.1.15 loglevel=ERROR status=500 user=admin
srcIP=10.0.0.12 loglevel=DEBUG status=302 user=user1
Step-by-Step
Starting with the source repository events
- flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] result{{Result Set}} repo --> 0 0 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;
Returns all events in which the status has a value starting with either
1
or2
. Notice that since the wildcard is used, the double-quotes is required.logscalein(status, values=["1*", "2*"])
Event Result set
The query is used to search status field for status codes starting with a given integer.
Sample output from the incoming example data:
srcIP | loglevel | status | user |
---|---|---|---|
10.0.0.1 | INFO | 200 | user1 |
Search two fields for multiple values in either first field or second field
case
{ in(srcIP, values=["10.1.168.2", "127.0.0.1"]);
in(targetIP, values=["10.0.0.1", "192.168.1.12"]); }
The in()
function can be used to select
events in which the given field contains specific values.
Sometimes it may be necessary to search for multiple values in two
different fields in the same query string. Though the
in()
function cannot directly be combined
with an OR clause, it is possible to use the
in()
function in a case statement to produce
the same output as an OR. In this example, the query will look for
events in either the srcIP field or the
targetIP.
Starting with the source repository events
- flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[/Filter/] result{{Result Set}} repo --> 0 0 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;
Filters for events in the srcIP field that contains the values
10.1.168.2
or127.0.0.1
and filters for events in the targetIP field that contains the values10.0.0.1
or192.168.1.12
. The returned results would be events from both fields. Notice that because it is a case statement, it executes and returns whether either field contains the corresponding values in the array.logscalecase { in(srcIP, values=["10.1.168.2", "127.0.0.1"]); in(targetIP, values=["10.0.0.1", "192.168.1.12"]); }
Event Result set
The query is used to query two fields for multiple/specific values in either first field or second field.