You may use this query function to format a string using
printf
style. The formatted
string is put in a new field. The input parameters or fields can
be one field or an array of fields.
Parameter | Type | Required | Default Value | Description |
---|---|---|---|---|
as | string | optional[a] | _format | The output name of the formatted field. |
field | array of strings | required | The fields to format. For multiple fields, enter within square brackets, separated by commas. | |
format [b] | string | required | The formatting codes for formatting the given string or strings. | |
timezone | string | optional[a] | The timezone when formatting dates and times. See Supported Time Zones for a list of supported timezones. | |
[a] Optional parameters use their default value unless explicitly set. |
Hide omitted argument names for this function
Omitted Argument NamesThe argument name for
format
can be omitted; the following forms of this function are equivalent:logscale Syntaxformat("value",field=["value"])
and:
logscale Syntaxformat(format="value",field=["value"])
These examples show basic structure only.
Fields can only be used as datetime values if they are
milliseconds since the beginning of the epoch (for example, 1
January 1970 00:00:00 UTC
).
format()
Syntax Examples
Since there are several fields and types of fields that may be
given with the format()
query function,
this section provides several examples of how to use the query
function.
As a first example, suppose you want to calculate a numeric value and want to format the results so that it shows only two decimal places. You would do that like this:
avg(field=cputime)
| format("%,.2f", field=_avg, as=_avg)
In this example, the query is averaging the field containing
the CPU value. This number is then piped to the
format()
function, which gives a
formatting code — how the field value should be
formatted; in the example, it formats the number to two
decimal places, using ,
as the
thousands separator.
_avg |
---|
288.582 |
Other examples of using the format()
can
be:
Concatenate two fields with a comma as separator:
format(format="%s,%s", field=[a, b], as="combined")
| table(combined)
Get the hour of day out of the event @timestamp:
format("%tm", field=@timestamp, as=hour)
| table(hour)
To format an integer adding a thousands separator:
count()
| format("%,i", field=_count, as=_count)
Produces
_count | ||
---|---|---|
3 | 886 | 817 |
Create a link with title based on the extracted content:
$extractRepo()
| top(repo)
| format("[Link](https://example.com/%s)", field=repo, as=link)
Important
Do not include a newline if the format specification for a link, even for readability. The link will not be identified correctly and may not create a link in the generated event list.
Format Specifiers
Deprecated:1.58
The format specifiers %a
,
%A
and
%S
were deprecated in 1.58
and removed completely in 1.70.
A format specifier is formed like this:
%[argument_index][flags][width][.precision][length]type[modifiers]
Sections in square brackets, for example,
[flags]
, are optional and
can be omitted.
Important
The resulting format specifier may be a two-letter code
where type
identifies the type, and a second letter (the
modifier)
identifies the specific format. For example, with the time
specifier, %t
only
identifies the field to be formatted as a date or time
value. The output formate must use a letter from
Date/Time to
indicate the actual format. For example:
parseTimestamp(field=@timestamp)
| timeonly := format("%tr",field=[@timestamp])
Formats the @timestamp using
%tr
which is the
equivalent of the formatTime()
specifier %r
, which
outputs a full natrual time value, for example, as
%tI:%tM:%tS %Tp
, for
example, 1:30:00 PM
Supported Types
The supported type specifiers are the following.
Type | Output |
---|---|
d or i | Signed decimal integer |
b or B | Boolean (uppercase and lowercase, respectively) |
o | Signed octal |
x or X | Unsigned hexadecimal integer (lowercase and uppercase, respectively) |
f or F | Decimal floating point |
e or E | Scientific-notation (exponential) floating point |
g or G | Scientific or decimal floating point |
t or T | Date/Time (lowercase or uppercase, respectively) |
c or C | Single character (lowercase or uppercase, respectively) |
s | String of characters |
n | Newline character |
% | The format specifier %% will produce a single % |
Numbers
Type | Output |
---|---|
d or i | Signed decimal integer |
o | Unsigned octal |
x or X | Signed hexadecimal integer |
f or F | Decimal floating point |
e or E | Scientific-notation (exponential) floating point |
g or G | Scientific or decimal floating point (depending on input) |
A number is a field that contains only an integer or a
real number, with no grouping, for example,
1,000
is not a number.
Scientific notation, for example,
1.74587E100
is supported.
If a field is not a number following the above
description, the output of
format given any
of the above type specifiers is
null
.
Octal Formatting
Type specifier o
, does
not support negative numbers. If the given field is a
negative integer, the output in undefined.
Hexadecimal Formatting
Type specifiers x
and
X
expect the
corresponding field to be that of a 64-bit signed
integer and produce the same integer in the
Hexadecimal
numeral system stripped of any leading zeros.
For instance, if a field
num contains the
value 42, running:
format("%X", field=[num])
The format()
function creates the
field _format =
2A. Notice that format()
does not add the common denomination of
0x
(from the C
programming language) to the produced output unless
given the #
flag.
Likewise, 0x
can be
added explicitly to the format string as:
format("0x%X", field=[num])
This would produce the field _format = 0x2A.
Hexadecimal formatting is closely related to the binary
representation of the integer, which is in
Two's
complement representation. This has the adverse
effect that the hexadecimals produced for negative
integers can have a large amount of leading
F
characters. If your
input is a signed 32-bit integer, you can shorten the
output of format()
down to only
display output corresponding to 32-bits, using the
length specifiers.
Floating Point Formatting
Type specifiers f
and
F
format the given field
as a real number with the specified precision. They only
work on floating point values up to
1e9
(one billion). See
Supported Precision
for more information.
Type specifiers e
and
E
formats the given
field as a real number in scientific notation, lowercase
and uppercase respectively. For instance,
176.54
formatted using
%e
becomes
1.765400e+02
.
For type specifiers g
and G
the specified
precision represents the amount of significant figures,
instead of the number of digits after the decimal point.
If the integer part of the number is larger than the
specified amount of significant digits,
g
and
G
behave like
e
and
E
respectively,
otherwise they behave like
f
and
F
. Notice that the
minimum precision is 6 and the maximum precision is 9.
Booleans (true and false)
Type | Output |
---|---|
b or B | Boolean |
On type specifier b
or
B
, if the corresponding
field is false
, then the
result is false
or
FALSE
respectively.
Otherwise, the result is
true
or
TRUE
, respectively.
Strings
Type | Output |
---|---|
c or C | Single character (lowercase or uppercase, respectively) |
s | String of characters |
n | Newline character |
A string is any sequence of length
>= 1
consisting of
unicode characters. Any field will match this description.
On type specifier c
, the
first character of the string is output. Type specifier
s
outputs the specified
field.
Date/Time
For dates and times, the format specifier
t
or
T
is a prefix for the
value specific format, as supported by the
formatTime()
and
formatDuration()
functions.
For example, to format the @timestamp
field to show the time in
HOUR:MINUTE
format using
format()
function, it would need the
format specifier %tH:%tM
:
parseTimestamp(field=@timestamp)
| timeonly := format("%tH:%tM",field=[@timestamp,@timestamp])
In the above example, note as well that the source field @timestamp must be provided twice to be decoded by each format specifier.
This is equivalent to using
formatTime()
:
parseTimestamp(field=@timestamp)
| timeonly := formatTime("%H:%M",field=@timestamp)
For more information, see
formatTime()
Fields can only be used as date/time values if they are in
milliseconds since the beginning of the Unix epoch, 1
January 1970 00:00:00 UTC. If the field is anything else,
format outputs null
.
All Date/Time type specifiers must be followed by a Date/Time modifier.
The following time modifiers are available:
Modifier | Output |
---|---|
H
| Hour of the day for the 24-hour clock, formatted as two digits with a leading zero as necessary, (00 - 23) |
I
| Hour of the day for the 12-hour clock, formatted as two digits with a leading zero as necessary, (01 - 12) |
k
| Hour of the day for the 24-hour clock, (0 - 23) |
l
| Hour of the day for the 12-hour clock, (1 - 12) |
M
| Minutes within the hour formatted as two digits with a leading zero as necessary, (00 - 59) |
S
| Seconds within the minute, formatted as two digits with a leading zero as necessary, that is 00 - 60 ('60' is a special value required to support leap seconds). |
L
| Milliseconds within the second formatted as three digits with leading zeros as necessary, (000 - 999) |
N
| Nanoseconds within the second, formatted as nine digits with leading zeros as necessary, (000000000 - 999999999) |
p
| Locale-specific morning or afternoon marker in lower case, for example 'am' or 'pm'. Use of the prefix 'T' forces this output to upper case. |
z
| RFC 822 style numeric time zone offset from GMT, for example -0800. This value will be adjusted as necessary for Daylight Saving Time. May depend on locale. |
Z
| A string representing the abbreviation for the time zone. This value will be adjusted as necessary for Daylight Saving Time. May depend on locale. |
s
|
Seconds since the beginning of the epoch starting at 1 January 1970 00:00:00 UTC, (Long.MIN_VALUE/1000 to Long.MAX_VALUE/1000 , where Long is a 64-bit signed integer.
|
Q
|
Milliseconds since the beginning of the epoch starting at 1 January 1970 00:00:00 UTC, (Long.MIN_VALUE to Long.MAX_VALUE , where Long is a 64-bit signed integer.
|
The following date modifiers are available:
Modifier | Output |
---|---|
Y
| Year, formatted as at least four digits with leading zeros as necessary, for example, 0092 equals 92 CE for the Gregorian calendar. |
y
| Last two digits of the year, formatted with leading zeros as necessary, that is 00 - 99. |
j
| Day of year, formatted as three digits with leading zeros as necessary, for example, 001 - 366 for the Gregorian calendar. |
m
| Month, formatted as two digits with leading zeros as necessary, that is 01 - 13. |
d
| Day of month, formatted as two digits with leading zeros as necessary, that is 01 - 31. |
e
| Day of month, formatted as two digits, that is 1 - 31. |
Furthermore, the following special date/time modifiers are available:
Modifier | Output |
---|---|
R
|
Time formatted for the 24-hour clock as H:M .
|
T
|
Time formatted for the 24-hour clock as H:M:S .
|
r
|
Time formatted for the 12-hour clock as I:M:S p . The location of the morning or afternoon marker (p ) may be locale-dependent.
|
D
|
Date formatted as m/d/y .
|
F
|
ISO 8601 complete date formatted as Y-m-d .
|
Other
Type | Output |
---|---|
% |
The format specifier %% will produce a single %
|
Supported Argument Index Specifiers
The argument index is a decimal integer indicating the
position of the argument in the fields list. The first
argument is referenced by
1$
, the second by
2$
, and so on. Another way
to reference arguments by position is to use the
'<'(\u003c)
flag, which
causes the argument for the previous format specifier to
be re-used. For example, the following two statements
produce identical strings:
format("Event date: %1$Tm/%1$Te/%1$TY", field=[@timestamp], timezone="Europe/Copenhagen")
format("Event date: %1$Tm/%<Te/%<TY", field=[@timestamp], timezone="Europe/Copenhagen")
If no argument index is specified, the first format specifier refers to the first argument of the fields list, the second format specifier refers to the second argument and so on.
Supported Flags
Flags | Description |
---|---|
-sign | Left-justify within the given field width; Right justification is the default. |
+sign | Forces preceding the result with a plus or minus sign (+ or -) even for positive numbers. By default, only negative numbers are preceded with a -sign. |
(space) | If no sign is written, a blank space is inserted before the value. |
# | Used with o, b, x or X type specifiers the value is preceded with 0, 0b, 0x or 0X respectively for values different than zero. Used with f or F it forces the written output to contain a decimal point even if no more digits follow. By default, if no digits follow, no decimal point is written. |
0 | Left-pads the number with zeros (0) instead of spaces when padding is specified (see width sub-specifier). |
, | Groups the output in thousands, for instance 10000 becomes 10,000. |
Supported Width
Width | Description |
---|---|
(number) | Minimum number of characters to be printed. If the value to be printed is shorter than this number, the result is padded with blank spaces. The value is not truncated even if the result is larger. |
Supported Precision
Precision | Description |
---|---|
.number | For integer specifiers (d, i, o, u, x, X): precision specifies the minimum number of digits to be written. If the value to be written is shorter than this number, the result is padded with leading zeros. The value is not truncated even if the result is longer. A precision of 0 means that no character is written for the value 0. For f and F specifiers: this is the number of digits to be printed after the decimal point. By default, this is 6, maximum is 9. For 'g' and 'G' specifiers: this is number of significant digits with which to display the number. For s: this is the maximum number of characters to be printed. By default all characters are printed until the ending null character is encountered. If the period is specified without an explicit value for precision, 0 is assumed. |
Supported Length
The length argument specifies the length with which to interpret the given fields' data type.
In general, format()
interprets any
number that is not a floating point number to be that of a
64-bit signed integer and formats any such integer with
leading zeros removed. For instance, converting 42 to
hexadecimal with the format string
0x%X
produces the string
0x2A
and not one with 62
leading zeros. However, conversions of negative numbers to
hexadecimal are represented using
Two's
complement which entails a large number of leading
F
characters. For example,
the number -1
is by
default represented using all 64-bits, hence the above
format string produces
0xFFFFFFFFFFFFFFFF
. This
can for example be brought down to
0xFFFFFFFF
by specifying
the h
length argument.
Length | Description |
---|---|
(none) | Signed 64-bit integer |
h | Signed 32-bit integer |
format()
Examples
Click
next to an example below to get the full details.Calculate the Mean of CPU Time
Calculate the sum of all numbers (mean) of the CPU time
Query
avg(field=cputimeNanos)
| cputime := (_avg/1000000)
| format("%,.2f", field=_avg, as=_avg)
Introduction
CPU time is the exact amount of time that the CPU has spent
processing data for a specific program or process. In this example
the avg()
function is used to calculate the
sum of all numbers; the mean of the CPU Time.
Step-by-Step
Starting with the source repository events.
- logscale
avg(field=cputimeNanos)
Calculates the mean of the field cputimeNanos. This can be run in the humio system repository to get the average time spent in nanoseconds for different activities. The mean is calculated by summing all of the values in the cputimeNanos field and dividing by the number of values across all events.
- logscale
| cputime := (_avg/1000000)
Calculates the average CPU time to milliseconds to make it easier to read.
- logscale
| format("%,.2f", field=_avg, as=_avg)
Overwrites the field _avg to contain the _avg field to show only two decimals.
Event Result set.
Summary and Results
The query is used to averaging the field containing the CPU value. This
value is then piped to the format()
function, which
provides a formatting code — how the field value should be
formatted. In this example, it formats the value to two decimal.
Calculation of CPU times is useful to determine processing power - for
example if troubleshooting a system with high CPU usage.
Sample output from the incoming example data:
_avg |
---|
0.14 |
Concatenate Fields and Strings Together
Query
format("%s/%s",field=[dirname,filename],as=pathname)
Introduction
The concat()
is not able to concatenate fields and
strings together. For example to create a pathname based on the
directory and filename it is not possible to do:
concat([dirname,"/",filename],as=pathname)
This will raise an error. Instead, we can use
format()
.
Step-by-Step
Starting with the source repository events.
- logscale
format("%s/%s",field=[dirname,filename],as=pathname)
Formats a value separating the two by a forward slash, creating the field pathname
Event Result set.
Summary and Results
The format()
function provides a flexible method of
formatting data, including encapsulating or combining strings and fields
together.
Format Values From Two Array Elements Using :
Format Values from Two Array Elements using : as a separator
Query
objectArray:eval("in[]", asArray="out[]", function={out := format("%s:%s", field=[in.key, in.value])})
Introduction
In this example, the objectArray:eval()
function is
used to format the array entries in[].key
and in[].value
separating the concatenated values with a
:
in the output field
out[]. The output must be a
single field in this example as format()
is only
capable of creating a single value.
Example incoming data might look like this:
in[0].key = x
in[0].value = y
in[1].key = a
in[1].value = b
Step-by-Step
Starting with the source repository events.
- logscale
objectArray:eval("in[]", asArray="out[]", function={out := format("%s:%s", field=[in.key, in.value])})
Iterates (executes a loop) over the array from start to end (or to the first empty index in the array), applies the given function, and returns the concatenated results in a new output array name field out[].
Notice that a
var
parameter can be used to give a different name to the input array variable inside the function argument. This is particularly useful whenever the input array name is very long. Example:logscaleobjectArray:eval("someVeryLongName[]", asArray="out[]", var=x, function={out := format("%s:%s", field=[x.key, x.value])})
Event Result set.
Summary and Results
The query is used to format arrays of objects.
Sample output from the incoming example data:
out[0] = x:y
out[1] = a:b
Perform Formatting on All Values in an Array
Perform formatting on all values in a flat array using the array:eval()
function
Query
array:eval("devices[]", asArray="upperDevices[]", var=d, function={upperDevices :=upper("d")})
Introduction
In this example, the array:eval()
function is used
to convert all values (for example
[Thermostat, Smart Light]
) in an
array devices[] from lowercase
to uppercase and show the results in a new array.
Example incoming data might look like this:
{\"devices\":[\"Thermostat\",\"Smart Plug\"],\"room\":\"Kitchen\"}" |
{\"devices\":[\"Smart Light\",\"Thermostat\",\"Smart Plug\"],\"room\":\"Living Room\"}" |
{\"devices\":[\"Smart Light\",\"Smart Plug\"],\"room\":\"Bedroom\"}" |
{\"devices\":[\"Thermostat\",\"Smart Camera\"],\"room\":\"Hallway\"}" |
{\"devices\":[\"Smart Light\",\"Smart Lock\"],\"room\":\"Front Door\"}" |
Step-by-Step
Starting with the source repository events.
- logscale
array:eval("devices[]", asArray="upperDevices[]", var=d, function={upperDevices :=upper("d")})
Formats all values in the array devices[] to uppercase and returns the results in a new array named upperDevices[]. The values in the original array stay the same:
[Thermostat, Smart Plug, Smart Light]
and the new array contains the returned results:[THERMOSTAT, SMART PLUG, SMART LIGHT]
Event Result set.
Summary and Results
The query is used to turn values in an array into uppercase. The
array:eval()
function can also be used for squaring
a list of numbers in an array.
Sample output from the incoming example data:
devices[] | upperDevices[] |
---|---|
Thermostat | THERMOSTAT |
Smart Plug | SMART PLUG |
Smart Light | SMART LIGHT |
Rounding to n Decimal Places
Query
format("%.2f", field=value)
Introduction
To round a number to a specific number of decimal points, use
format()
rather than
round()
Step-by-Step
Starting with the source repository events.
- logscale
format("%.2f", field=value)
Rounds the field value to two decimal places.
Event Result set.