This query function may be used to select events in which the given field
contains particular values. For instance, you might want to monitor events
in which log messages contain error, warning, or other similar words in
log entries, or perhaps particular numeric values in other fields.
Although this query function allows for only three parameters, it is very
useful and versatile. For the first parameter, you would specify the field
on which to filter data. The second parameter sets whether the search
should be case-insensitive. The third parameter would be the string or
multiple strings on which to match the contents of the field.
Suppose you have a repository which is ingesting data from a few web
servers. And suppose that you want to get a list of events in which the
user received the HTTP code 404, for web pages
Not Found. You could do that easily with this
query:
logscale
status=404
As this suggests, the field on which to check is
status.
Suppose further that you want to get a list of events in which the user
received the HTTP codes 422 and
200. Those codes represent respectively
Unable to be processed, and
Successful. You could get those events with the
in() function like so:
logscale
in(status,values=["422","200"])
Using the status field for the
first parameter; for the second parameter, the two statuses are listed,
separated by commas, within an array — within square-brackets.
Incidentally, if you wanted to include string values instead of numbers,
each string value would have to be contained within double-quotes.
There are a few other HTTP codes related to errors besides these two.
You could list all of them in the array, or you could add the wildcard
(for example, *) like this:
logscale
in(status,values=["4*"])
This will return all events in which the
status has a value starting with
4. Notice that even though only one value is given, you have to include
the square-brackets. Also, notice that since the wildcard is used, the
double-quotes is required.
Using the field parameter in addition to the
=~ syntax:
The in() function can be used to select
events in which the given field contains specific values. It is
possible to combine the in() with the
if() function to categorize log level errors
and their criticality. In this more advanced example, the
if() function is used to categorize errors
based on a time condition and it compares the status of a log
level and decides on the log's criticality. The field
critical_status is going
to be evaluated based on the if() function.
Searches for events where the field
status contains the
values 500 or
400 and assigns the value
Critical to a field named
critical_status for the
returned results. If the values are not equal to
500 or
400, then the returned
events will have the value
Non-Critical assigned to
the field
critical_status.
Event Result set.
Summary and Results
The query is used to categorize errors in log levels according
to their criticality.
Sample output from the incoming example data:
srcIP
loglevel
status
user
critical_status
192.168.1.5
ERROR
404
admin
Critical
10.0.0.1
INFO
200
user1
Non-Critical
172.16.0.5
WARN
422
user2
Non-Critical
192.168.1.15
ERROR
500
admin
Critical
10.0.0.12
DEBUG
302
user1
NonCritical
Categorize Events Based on Values in More Fields
Categorize events based on values across multiple fields - the example uses a combination of in() with case, match(), and if()
The in() function can be used to select
events in which the given field contains specific values. It is
possible to combine the in() with a case
statement to categorize events. In this more advanced example, a
case statement is used to categorize events based on the fields
srcIP and
loglevel, using both
in() and negated in().
Notice that the semi-colon is used to end the different logical
expressions.
Returns all events with values starting with
192.168.1.* followed by
anything in the scrIP
field and then creates a new field named
type with the assigned
value Internal for the
returned results. Notice that since the wildcard is used, the
double-quotes is required. Next, the query searches for events
where the field loglevel
does not contain the values
DEBUG or
INFO and assigns the value
Critical to the returned
results in the type
field. For anything else, it sets the value in the
type field to
Other. In this example,
INFO and
DEBUG will therefore be
set to Other. The above
case statement can also be expressed like this: If the sourceIP
equals the value
192.168.1.* followed by
anything, then identify the
type field as
Internal. If it is not
equal to the loglevel of debug or info, then identify the
type field as
Critical. If it does not
match either of the above, identify the
type field as
Other.
Event Result set.
Summary and Results
The query is used to to categorize events and define their type.
Sample output from the incoming example data:
srcIP
loglevel
status
user
type
192.168.1.5
ERROR
404
admin
Internal
10.0.0.1
INFO
200
user1
Other
172.16.0.5
WARN
422
user2
Critical
192.168.1.15
ERROR
500
admin
Internal
10.0.0.12
DEBUG
302
user1
Other
Differentiate Between Types of Log Levels
Differentiate between types of log levels using the in() function with the match expression
The in() function can be used to select
events in which the given field contains specific values. It is
possible to combine the in() with a match
expression to differentiate between the different types of log
levels. In this more advanced example, we match against the
loglevel using the match filter statement. Notice that the
semi-colon is used to end the different logical expressions.
Matches all log levels which have the value/word
ERROR inside their dataset
and creates a new field named
severity with the
assigned value High for
the returned results/matches. Then it matches events with the
values DEBUG or
INFO and assigns the value
Low to the returned
results in the severity
field. If the severity
field does not exist, it will create it, if the
severity field does
exist, it will overwrite the value of the field. For anything
else, it sets the value in the
severity field to
Medium. In this example, a
loglevel like WARN will
therefore be set to
Medium. Notice the use of
double-quotes around the values to right of the assignment
operator, if not used, it will be interpreted as a field and not
a string.
Event Result set.
Summary and Results
The query is used to differentiate between types of log levels.
Sample output from the incoming example data:
srcIP
loglevel
status
user
severity
192.168.1.5
ERROR
404
admin
High
10.0.0.1
INFO
200
user1
Low
172.16.0.5
WARN
422
user2
Medium
192.168.1.15
ERROR
500
admin
High
10.0.0.12
DEBUG
302
user1
Low
Exclude Events With Specific Values From Searches
Exclude events with specific values from searches using the negated function in()
Query
logscale
!in(loglevel,values=["ERROR","WARN"])
Introduction
The in() function can be used to select
events in which the given field contains specific values. It is
possible to exclude events with specific values using the
in() function with a negation in front. In
this example, events will be excluded from the search result if
the loglevel field
contains the values ERROR or
WARNING.
The in() function can be used to select
events in which the given field contains specific values. It is
possible to perform case-insensitive searches on a field using the
in() function. In this example, the
loglevel field is searched
for occurrences of either
error or
warning.
Returns all events in which the
loglevel field has the
value error or
warning. As it is
case-insensitive, it returns all occurences of the specified
values in all their variants, regardless of the case.
Event Result set.
Summary and Results
The query is used to perform case-insensitive searches on a
specific value in a given field. This is useful when searching
for strings where values may appear in both both upper and lower
case to ensure that all events are extracted.
Sample output from the incoming example data:
srcIP
loglevel
status
user
192.168.1.5
ERROR
404
admin
172.16.0.5
WARN
422
user2
192.168.1.15
ERROR
500
admin
Search Single Field for Multiple Values
Search single field for multiple values using the in() function
Query
logscale
in(status,values=["404","422"])
Introduction
The in() function can be used to select
events in which the given field contains specific values. In this
example, the in() function is used to search
for events in which the user received the HTTP codes
404 and
422.
Searches for events with the values
404 and
422 in the
status field. If you
want to search for all values starting with 4, it is possible to
just use the query in(status, values=["4*"]) - here
it is important to remember the double-quotes because of the
wildcard usage.
Event Result set.
Summary and Results
The query is used to search a single field for specific values.
This is useful when monitoring events in which log messages
contain error, warning, or other similar words in log entries,
or perhaps specific numeric values in other fields. In this
example, it selects logs with specific HTTP statuses. If you
just want to search a single field for one specific value, use
this query: status = 404 instead of the
in() function.
Sample output from the incoming example data:
srcIP
loglevel
status
user
192.168.1.5
ERROR
404
admin
172.16.0.5
WARN
422
user2
Search Status Field for All Status Codes Starting With "1" or "2"
Use a wildcard with in() to select all status codes starting with 1 or 2
Query
logscale
in(status,values=["1*","2*"])
Introduction
The in() function can be used to select
events in which the given field contains specific values. It is
possible to use wildcards with the in()
function to select for example all status codes starting with "1"
or "2". Notice that "" must
be used around the *.
The in() function can be used to select
events in which the given field contains specific values.
Sometimes it may be necessary to search for multiple values in two
different fields in the same query string. Though the
in() function cannot directly be combined
with an OR clause, it is possible to use the
in() function in a case statement to produce
the same output as an OR. In this example, the query will look for
events in either the srcIP
field or the targetIP.
Filters for events in the
srcIP field that
contains the values
10.1.168.2 or
127.0.0.1 and filters for
events in the targetIP
field that contains the values
10.0.0.1 or
192.168.1.12. The returned
results would be events from both fields. Notice that because it
is a case statement, it executes and returns whether either
field contains the corresponding values in the array.
Event Result set.
Summary and Results
The query is used to query two fields for multiple/specific
values in either first field or second field.