New fields can be created in two ways:
Regex Field Extraction
You can extract new fields from your text data using regular expressions and then test their values. This lets you access data that Humio did not parse when it indexed the data.
For example, if your log entries contain text such as
disk_free=2000 ..., then you can use a query like the
following to find the entries that have less than 1000 free disk space
regex("disk_free=(?<space>[0-9]+)") | space < 1000
Named capturing groups are used to extract fields in regular
expressions. The field
space is extracted and is then
available after the regex function. The same can be written using a
@rawstring=/disk_free=(?<space>[0-9]+)/ | space < 1000
You can apply repeat to field extraction to yield one event for each match of the regex. This allows processing multiple values for a named field, or a field name that matches a pattern, as in this example:
regex("value[^=]*=(?<someBar>\\S+)", repeat=true) | groupby(someBar)
On an input event with a field value of
valueExtra=bar2 value=bar3 the
sees all three bar values.
In order to use field-extraction this way, the regex must be a
top-level expression, that is,
| between bars
|. The following doesn't work
// DON'T DO THIS - THIS DOES NOT WORK type=FOO or /disk_free=(?<space>[0-9]+)/ | space < 1000
Since regular expressions do need some computing power, it is best to do as much simple filtering as possible earlier in the query chain before applying the regex function. (For version 1.13+: If the field name is known it also helps to specify the field to search to avoid searching all fields.)
Fields can also be added by functions. Most functions set their result
in a field that has the function name prefixed with a
_ by default. For example the
count() puts its result in a field
Most functions that produce fields have a parameter called
as. By setting this parameter you can specify the
name of the output field, for example
Assigns the result of the
count() to the field
cnt (instead of the default
See also the Assignment Operator for shorthand syntax for assigning results to a field.
eval() can assign fields while doing
numeric computations on the input.
:= syntax is short for eval. Use
| between assignments.
... | foo := a + b | bar := a / b | ...
This is short for the following:
... | eval(foo = a + b) | eval(bar = a / b) | ...
You can use the operator
:= with functions that take
as parameter. When what's on the right hand side
of the assignment is a function call, the assignment is rewritten to
as= argument which, by convention, is the
output field name. For example
... | foo := min(x) | bar := max(x) | ...
The above is short for this:
... | min(x, as=foo) | max(x, as=bar) | ...
You can use
attr =~ fun() with any function that has
a parameter named
field. It designates the
field=attr argument and lets you write:
... | ip_addr =~ cidr(subnet="127.0.0.1/24") | ...
The alternative is this:
... | cidr(subnet="127.0.0.1/24", field=ip_addr) | ...