Conditional Evaluation
In LogScale the streaming style is not well-suited for procedural-style conditions. However, there are a few ways to do conditional evaluation:
Case Statements
Using case
statements, you can define alternative flows in
your queries. It is similar to case
or
cond
you might know from many other
functional programming languages.
The syntax looks like this:
case {
pipeline1
| ... ;
pipeline2
| ... ;
pipeline3
| ... ;
*
| ...
}
Each clause in the pipeline is terminated by a semicolon. For each
clause within the case
statement, the pipeline is run
individually as if the expression were executed within LogScale
as a typical query. If the expressions return events, the clause is
considered to have matched.
Typically, each clause takes the form of one or more test expressions followed by one or more results. For example, a single test and result:
src="client-side"
| type:="client"
The first statement tests for a field value, the second creates a new field.
OR for a multiple statement test:
src="client-side"
| ip != "127.0.0.1"
| type:="local"
In this example we are testing both the field src and ip.
The pipeline is considered to have matched if the pipeline returns events. If the test returns true and a value is set, then the clause has matched. If no clauses match, then events are dropped.
You can add a wildcard or default clause that will be used if no other clauses match by using an asterisk in the pipeline:
case { ... ;
*
| DEFAULT }
The default matches all events in the "default case", performing similar
to the else
part of an if-statement. If
you do not add a wildcard clause, any events that do not match any of
the explicit clauses will be dropped.
Conditional Case Example
Let's say we have logs from multiple sources that all have a field
named time
, and we want to group those hosts by a
type identifying what host group to produce percentiles of the time
fields, but one for each kind of source.
First, we try to match some text that distinguishes the different
types of line. Then we can create a new field type
and assign a value that we can use to group by.
time=*
| case {
src="client-side"
| type := "client";
src="frontend-server"
| ip != 192.168.1.1
| type := "frontend";
*
| type := "server"
}
| groupBy(type, function=percentile(time))
Within the case
statement, there are three separate
clauses:
- logscale Syntax
src="client-side" | type := "client";
When the src field contains
client-side
, set the type field toclient
. - logscale Syntax
src="frontend-server" | ip != 192.168.1.1 | type := "frontend";
When the src field contains
frontend-server
and provided that the ip is not192.168.1.1
, set the type field tofrontend
. - logscale Syntax
* | type := "server"
If no other clause matches, set type to
server
.
Catch-All Clause
The *
clause captures any output that
has not matched a previous clause, and can be used either to ignore
that information (resulting in no action or output) or to apply a
default operation or value. For example, in the sample below the
client
field is being used to
determine whether the IP address is
localhost
and setting the
local
field accordingly:
case { client = "::1"
| local := "true";
client = "127.0.0.1"
| local := "true";
*
| local := "false"}
The *
clause in this instance sets
local
to false
for
any non-matching value.
However, the following is also valid:
case { client = "::1"
| local := "true";
client = "127.0.0.1"
| local := "true";
* }
The above ensures that the non-matching clauses are not processed, but does not create the field unless we have identified a local value.
The following example demonstrates a base match and default response example when looking at the Event_SimpleName field:
Event_SimpleName match{
/.*Network.*/ => network_tag := "Network";
* => new_tag := "Other"}
Match Statements
If you you looking for the
match()
function, see
match()
Using match
statements, you can describe
alternative flows in your queries where the conditions all check the
same field. It is similar to the
switch
operation you might
recognize from many other programming languages. The matches on the
field support the filters listed in
Field Filters and in
Regular Expression Filters.
The syntax looks like this:
field match {
value => expression
| expression... ;
/regex/ => expression
| ...;
* => expression
| ...
}
You write a sequence of filter and pipeline clauses to run when the
filter matches, separated by a semicolon
(;
). LogScale will apply each
clause from top to bottom until one returns a value (matches the input).
You can use some functions as selectors (in addition to string patterns). More specifically, those functions which test a single field (and do not transform the event).
You can add a wildcard clause match { ... ; *
=> * }
which matches all events as the "default case",
essentially the else
part of an
if-
statement. If you do not add a
wildcard clause, then any events that do not match any of the explicit
clauses will be dropped. You cannot use the empty clause — you
must explicitly write *
to match all.
As opposed to how a regular field test would work — where
x=*
checks for existence and
x=true
checks that x
matches the
string true
— in the match (for case
statements) the guards *
and
true
are equivalent and matches everything.
Because *
no longer has the same
meaning as in field tests, x match { ** =>
*}
and x match { * => *}
are not equivalent since the first checks for existence and the latter
accepts everything.
This also means that "*"
and
*
are not equivalent either, and
"*"
and
"**"
are equivalent.
Since, as in case statements, the guards
*
and
true
match anything, the existence
of a field can be checked using a quote glob
"*"
or two consecutive globs
**
.
The syntax looks like this:
field match {
value => expression
| expression... ;
/regex/ => expression
| ...;
in(values=[1, 2, 3]) => expression
| ...; // match if in(field=field, values=[1, 2, 3]) does
"*" => expression
| ...; // match if the field exists
* => expression
| ... // match all events
}
match
statements accept any "pure"
Filtering Query Functions (filter functions that do not mutate)
that has a field
parameter, for example:
x match { in(values=[5, 56, 567]) => *;}
Conditional Match Example
Let's say we have logs from multiple sources that all have a field that holds the time spent on some operation, but in different fields and units. We want to get percentiles of the time fields all in the same unit and in one field.
logtype match {
"accesslog" => time:=response_time ; // Access log is in seconds.
/server_\d+/ => time:=server_time*1000 ; // These servers log in millis
}
| groupBy(logtype, function=percentile(time))
Setting a Field's Default Value
You can use the function default()
to set the
value of a missing or empty field or use the function
coalesce()
to select the first defined value from a list of
expressions.