Query Functions
Manipulating and formatting functions for extracting information from your event data
LogScale query functions take a set of events, parameters, or configurations. They produce, reduce, or modify values within that set, or in the events themselves within a query pipeline.
Below is an alphabetical listing of all LogScale query functions:
Table: Query Functions
| Function | Type | Default Argument | Availability | Description |
|---|---|---|---|---|
array:contains(array, value) | array, data-manipulation, filter | array | Checks whether the given value matches any of the values of the array and excludes the event if no value matches | |
array:filter(array, [asArray], function, var) | array, data-manipulation | array | Drops entries from the input array using the given filtering function. | |
array:length(array, [as]) | array, array-length | array | Counts the number of elements in an array. | |
array:reduceAll(array, function, var) | aggregate, array, data-manipulation | array | Computes a value from all events and array elements of the specified array. | |
array:regex(array, [flags], regex) | array, filter, regular-expression | array | Checks whether the given pattern matches any of the values of the array and excludes the event from the search result | |
asn([as], [field]) | network | field | Determines autonomous system number and organization associated. | |
avg([as], field) | aggregate, statistics | field | Calculates the average for a field of a set of events. | |
base64Decode([as], [charset], field) | hash, parsing | field | Performs Base64 decoding of a field. | |
bitfield:extractFlags(field, output) | data-manipulation | Interprets an integer as a bit field and extracts the specified flags | ||
bucket([buckets], [field], [function], [limit], [minSpan], [span], [timezone], [unit]) | aggregate | span |
Extends the groupBy() function for grouping
by time.
| |
callFunction([as], field, function) | aggregate | function | Calls the named function on a field over a set of events. | |
cidr([column], field, [file], [negate], [subnet]) | filter, network | field | Filters events using CIDR subnets. | |
collect(fields, [limit], [multival], [separator]) | aggregate | fields | Collects fields from multiple events into one event. | |
communityId([as], destinationip, [destinationport], [icmpcode], [icmptype], proto, [seed], sourceip, [sourceport]) | network | Computes the Community ID, a standard for hashing network flows. | ||
concat([as], field) | data-manipulation, string | field | Concatenates the values of a list of fields into a value in a new field. | |
concatArray([as], field, [from], [prefix], [separator], [suffix], [to]) | array, data-manipulation, string | field | Concatenates values of all fields with same name and an array suffix into a new field. | |
copyEvent(type) | event | type | Duplicates event so pipeline will see both events. | |
count([as], [distinct], [field]) | aggregate, statistics | field | Counts given events. | |
counterAsRate([as], field) | aggregate | field | Calculates the rate for a counter field. | |
createEvents(rawstring) | aggregate-testing, event | rawstring | Generates temporary events as part of the query. | |
crypto:md5([as], field) | hash | field | Computes a cryptographic MD5-hashing of an input string | |
default(field, [replaceEmpty], value) | event, format | value | Creates a field to given parameter and given value. | |
drop(fields) | event | fields | Removes attributes or columns from a result set. | |
dropEvent() | event | Drops completely an event in parser pipeline to stop it from being ingested. | ||
duration([as], duration) | event | duration | Computes the number of milliseconds denoted by a human-readable specification. | |
end([as]) | time-date | as |
Assign the end of the search time interval to the field provided
by parameter as.
| |
eval() | data-manipulation | Creates a new field by evaluating the provided expression. | ||
eventFieldCount([as]) | event | Computes number of fields event uses internally for the values. | ||
eventInternals([prefix]) | event | Add a set of fields describing the storage locations of this event. | ||
eventSize([as]) | event | Determines the number of bytes that this event uses internally for the values, not counting the bytes for storing the field names. | ||
fieldset() | event | Retrieves a list of available fields. | ||
fieldstats([limit]) | aggregate, event | Retrieves stats about fields. | ||
findTimestamp([addErrors], [as], [field], [timezone], [timezoneAs]) | time-date | Finds timestamp in given field and parses, trying multiple timestamp formats. | ||
format([as], field, format, [timezone]) | data-manipulation, format | format | Formats a string using printf-style. | |
formatDuration([as], field, [from], [precision]) | format | field | Formats a duration into a more readable string. | |
formatTime(as, [field], format, [locale], [timezone]) | format | format |
Formats a string according to
strftime().
| |
geography:distance([as], lat1, lat2, lon1, lon2) | geolocation | introduced in 1.134.0 | Calculates the distance between two geographical coordinates along an ideal earth surface | |
geohash([as], [lat], [lon], [precision]) | geolocation | Calculates a geohash value given two fields representing latitude and longitude. | ||
getField([as], source) | getfield | Reads dynamically-named fields that are computed from an expression. | ||
groupBy(field, [function], [limit]) | aggregate | field | Groups events by specified fields and executes aggregate functions on each group. | |
hash([as], field, [limit], [seed]) | event, hash | field | Computes a non-cryptographic hash of a list of fields. | |
hashMatch([bits], [field], [hash], input, [salt]) | security | input | Calculates a secure hash of a field and uses it to match events as a filter. | |
hashRewrite([as], [bits], field, [hash], [replaceInRawstring], salt) | security | field | Calculates a secure hash of a field for storing in the event. | |
head([limit]) | aggregate | limit | Finds the oldest events. | |
if([as], condition, else, then) | condition | condition | Computes a value in one of two ways (evaluates one of two expressions). | |
in(field, ignoreCase, values) | comparison, filter | field | Filters records by values where field is in given values. | |
ioc:lookup([confidenceThreshold], field, [include], [prefix], [strict], type) | security | field | Look up IOCs (Indicators of Compromise). | |
ipLocation([as], [field]) | geolocation | field | Determines country, city, longitude, and latitude for given IP address. | |
join([end], field, [include], [key], [limit], [live], [max], [mode], query, [repo], [start], [view]) | join | query | Join two LogScale searches. | |
json:prettyPrint([as], [field], [step], [strict]) | data-manipulation, format | field | Nicer output to a JSON field. | |
kvParse([as], [excludeEmpty], [field], [OnDuplicate], [override], [separator], [separatorPadding]) | parsing | field | Key-value parse events. | |
length([as], field) | string | field | Computes the number of characters in a string field. | |
linReg([prefix], x, y) | aggregate | Computes linear relationship model between two variables using least-squares fitting. | ||
lower([as], field, [locale], [type]) | format | field | Changes text of a given string field to lower-case letters. | |
lowercase(field, [include], [locale]) | data-manipulation, format, string | field | Changes field name or content to lowercase for parsers. | |
match([column], field, file, [glob], [ignoreCase], [include], [mode], [strict]) | filter, string | file | Searches text using a CSV or JSON file and can enhance entries. | |
math:abs([as], field) | math | field | Calculates the absolute value of a field; the result is always a positive number or 0. | |
math:arccos([as], field) | math | field | Calculates the arc cosine of a field. | |
math:arcsin([as], field) | math | field | Calculates the arc sine of a field. | |
math:arctan([as], field) | math | field | Calculates the arc tangent of a value. | |
math:arctan2([as], x, y) | math | Calculates the arc tangent of a value. | ||
math:ceil([as], field) | math | field | Rounds field value to smallest integer that's larger than or equal to it. | |
math:cos([as], field) | math | field | Calculates the cosine of a field. | |
math:cosh([as], field) | math | Computes the hyperbolic cosine of a double field. | ||
math:deg2rad([as], field) | math | field | Converts angles from degrees to radians. | |
math:exp([as], field) | math | field | Calculates Euler's number e raised to the power of a double value in a field. | |
math:expm1([as], field) | math | field | Calculates the exponential value of a number minus 1. | |
math:floor([as], field) | math | field | Computes the largest integer value not greater than the field value given. | |
math:log([as], field) | math | field | Calculates the natural logarithm (base e) of the value in a double field. | |
math:log10([as], field) | math | field | Calculates the base 10 logarithm of a double field. | |
math:log1p([as], field) | math | field | Calculates the natural logarithm of the sum of field's value and 1. | |
math:log2([as], field) | math | field | Calculates the base 2 logarithm of a double field. | |
math:mod([as], divisor, field) | math | field | Calculates the floor modulus of field value and the divisor. | |
math:pow([as], exponent, field) | math | field | Calculates the field value to the exponent power. | |
math:rad2deg([as], field) | math | field | Converts angles from radians to degrees. | |
math:sin([as], field) | math | field | Calculates the sine of a field. | |
math:sinh([as], field) | math | field | Calculates the hyperbolic sine of a double field. | |
math:sqrt([as], field) | math | field | Calculates the rounded positive square root of a double field. | |
math:tan([as], field) | math | field | Calculates the trigonometric tangent of an angle in a field. | |
math:tanh([as], field) | math | field | Calculates the hyperbolic tangent of a field. | |
max([as], field, [type]) | statistics | field | Finds the largest number for the specified field over a set of events. | |
min([as], field, [type]) | statistics | field | Finds the smallest number for the specified field over a set of events. | |
now([as]) | time-date | as |
Assign the current time to the field provided by parameter
as.
| |
parseCEF([field], [headerprefix], [keeplabels], [labelprefix], [prefix]) | data-manipulation, parsing | field | Parses CEF version 0.x encoded messages. | |
parseCsv(columns, [delimiter], [excludeEmpty], field) | data-manipulation, parsing | field | Parses a CSV-encoded field into known columns. | |
parseFixedWidth(columns, [field], [trim], widths) | parsing | field | Parses a fixed width-encoded field into known columns. | |
parseHexString([as], [charset], field) | data-manipulation, math, parsing | field | Parses input from hex encoded bytes, decoding resulting bytes as a string. | |
parseInt([as], [endian], field, [radix]) | math, parsing | field | Converts an integer from any radix or base to base-ten, decimal radix. | |
parseJson([exclude], [excludeEmpty], field, [handleNull], [include], [prefix], [removePrefixes]) | data-manipulation, parsing | field | Parses specified fields as JSON. | |
parseLEEF([delimiter], [field], [headerprefix], [keeplabels], [labelprefix], [parsetime], [prefix], [timezone]) | data-manipulation, parsing | field | Parses LEEF version 1.0 and 2.0 encoded messages. | |
parseTimestamp([addErrors], [as], [caseSensitive], field, [format], [timezone], [timezoneAs]) | parsing, time-date | format | Parses a string into a timestamp. | |
parseUri([defaultBase], field, [prefix]) | parsing | field | Extracts URI components from a field. | |
parseUrl([as], [field]) | parsing | field | Extracts URL components from a field. | |
parseXml(field, [prefix], [strict]) | parsing | field | Parses specified field as XML. | |
percentile([accuracy], [as], field, [percentiles]) | aggregate, statistics | field | Finds one event with a field for each percentile specified. | |
range([as], field) | statistics | field | Finds numeric range between smallest and largest numbers for field over a set of events. | |
rdns([as], field, [server]) | aggregate, network | field | Events using RDNS lookup. | |
readFile(file, [include], [limit]) | data-manipulation, event | file | Uses a .csv lookup file as data input for the query. | |
regex([field], [flags], [limit], regex, [repeat], [strict]) | filter, regular-expression, string | regex | Extracts new fields using a regular expression. | |
rename([as], field) | data-manipulation, event | field | Renames one or more given fields. | |
replace([as], [field], [flags], regex, [replacement], [with]) | data-manipulation, regular-expression, string | regex | Replaces each substring that matches given regular expression with given replacement. | |
round([as], field, [how]) | math | field | Rounds an input field up or down, depending on which is nearest. | |
sample([field], [percentage]) | filter, statistics | percentage | Samples the event stream. | |
sankey(source, target, [weight]) | aggregate, data-manipulation, widget | Produces data compatible with Sankey widget. | ||
select(fields) | event | fields | Used to specify a set of fields to select from each event. | |
selectFromMax(field, include) | aggregate | field | Selects event with the largest value for the specified field. | |
selectFromMin(field, include) | aggregate | field | Selects event with the smallest value for the specified field. | |
selectLast(fields) | aggregate | fields | Specify fields to select from events, keeping value of most recent event for each field. | |
selfJoin([collect], field, [limit], [postfilter], [prefilter], [select], where) | join | field | Used to collate data from events that share a key. | |
selfJoinFilter(field, [prefilter], where) | filter, join | field | Runs query to determine IDs, and then gets all events containing one of them. | |
series(collect, [endmatch], [maxduration], [maxpause], [memlimit], [separator], [startmatch]) | aggregate, data-manipulation | collect | Collects a series of values for selected fields from multiple events into one or more events. | |
session([function], [maxpause]) | aggregate, statistics | function | Collects events into sessions, and aggregates them. | |
setField(target, value) | aggregate | Sets fields whose names are not known but computed from an expression. | ||
shannonEntropy([as], field) | network | field | Calculates a entropy measure from a string of characters. | |
sort([field], [limit], [order], [reverse], [type]) | aggregate | field | Sorts events by their fields. | |
split([field], [strip]) | array, data-manipulation, regular-expression, string | field | Splits an event structure created by a JSON array into distinct events. | |
splitString([as], by, [field], [index]) | array, data-manipulation, string | field | Splits a string by specifying a regular expression by which to split. | |
start([as]) | time-date | as |
Assign the start of the search time interval to the field provided
by parameter as.
| |
stats([function]) | aggregate, statistics | function | Used to compute multiple aggregate functions over the input. | |
stdDev([as], field) | statistics | field | Calculates the standard deviation for a field over a set of events. | |
stripAnsiCodes([as], field) | data-manipulation, string | field | Removes ANSI color codes and movement commands. | |
subnet([as], bits, field) | network | field | Computes a subnet from a IPV4 field. | |
sum([as], field) | aggregate | field | Calculates the sum for a field over a set of events. | |
table(fields, [limit], [order], [reverse], [sortby], [type]) | aggregate, widget | fields | Used to create a widget to present the data in a table. | |
tail([limit]) | aggregate | limit | Finds the newest events. | |
test(expression) | comparison, event, filter | expression | Evaluates boolean expression and filters events. | |
time:dayOfMonth([as], [field], [timezone], [timezoneField]) | time-date | field | Gets the day of the month of a timestamp field. | |
time:dayOfWeek([as], field, [timezone], [timezoneField]) | time-date | field | Gets day of week from 1 (Mon) to 7 (Sun) of a timestamp. | |
time:dayOfWeekName([as], [field], [timezone], [timezoneField]) | time-date | field | Gets the English display name of day of the week of a timestamp field. | |
time:dayOfYear([as], [field], [timezone], [timezoneField]) | time-date | field | Gets the day of the year of a timestamp field, from 1 to 365, or 366 in a leap year. | |
time:hour([as], [field], [timezone], [timezoneField]) | time-date | field | Gets the hour (24-hour clock) of a timestamp field. | |
time:millisecond([as], [field], [timezone], [timezoneField]) | time-date | field | Gets the millisecond of a timestamp field. | |
time:minute([as], [field], [timezone], [timezoneField]) | time-date | field | Gets the minute value of a timestamp field. | |
time:month([as], [field], [timezone], [timezoneField]) | time-date | field | Gets the month of a timestamp field (from 1 to 12). | |
time:monthName([as], [field], [timezone], [timezoneField]) | time-date | field | Gets the English name of month of a timestamp field (e.g., January). | |
time:second([as], [field], [timezone], [timezoneField]) | time-date | field | Gets the second of a timestamp field. | |
time:weekOfYear([as], [field], [timezone], [timezoneField]) | time-date | field | Gets the week number within a year of a timestamp, a value from 1 to 53. | |
time:year([as], [field], [timezone], [timezoneField]) | time-date | field | Gets the year of a timestamp field. | |
timeChart([buckets], [function], [limit], [minSpan], [series], [span], [timezone], [unit]) | aggregate, widget | series | Used to draw a linechart where the x-axis is time. | |
tokenHash([as], field) | hash, string | field | Calculates a structure hash which is equal for similarly structured input. | |
top([as], [error], field, [limit], [max], [percent], [rest], [sum]) | aggregate | field | Finds the top results based on a given field. | |
transpose([column], [header], [limit], [pivot]) | aggregate, data-manipulation, string | pivot | Transposes a query results set by creating an event for each attribute. | |
unit:convert([as], [binary], field, [from], [keepUnit], [to], [unit]) | data-manipulation, math | field | Converts values between different units. | |
upper([as], field, [locale]) | data-manipulation, format | field | Changes contents of a string field to upper-case letters. | |
urlDecode([as], field) | network | field | URL-decodes the contents of a string field. | |
urlEncode([as], field, [type]) | data-manipulation, network | field | URL-encodes the contents of a string field. | |
wildcard([field], [ignoreCase], pattern) | filter | pattern | Performs a wildcard pattern search with optional case insensitivity. | |
window([buckets], [function], [span]) | aggregate | function | Computes aggregate functions over a sliding window of data. | |
worldMap([ip], [lat], [lon], [magnitude], [precision]) | aggregate, widget | Used to produce data compatible with the World Map widget. | ||
writeJson([as], [field]) | data-manipulation, format | field | Writes data, including fields, as a JSON object. | |
xml:prettyPrint([as], field, [step], [strict], [width]) | data-manipulation | field | Nicer output to an XML field. |