Query Functions
LogScale query functions take a set of events, parameters, or configurations. They produce, reduce, or modify values within that set, or in the events themselves within a query pipeline.
The Query Functions table is an alphabetical listing of all LogScale query functions:
Table: Query Functions
| Function | Type | Default Argument | Availability | Description |
|---|---|---|---|---|
accumulate([current], function) | aggregate, data-manipulation, sequence | function | Â | Applies an aggregation function cumulatively to a sequence of events. |
array:append(array, values) | array, data-manipulation | array | Â | Appends single or multiple values to an array, or creates a new array if it does not already exist. |
array:contains(array, value) | array, data-manipulation, filter | array | Â | Checks whether the given value matches any of the values of the array and excludes the event if no value matches. |
array:dedup(array, [asArray]) | array, data-manipulation | array | Â |
array:dedup() removes duplicate elements from
an array. The ordering of the first occurrence of each unique
element is preserved.
|
array:drop(array) | array | array | Â | Takes the name of an array and drops all fields of this array. |
array:eval(array, [asArray], function, [var]) | array | array | Â | Evaluates the function argument on all values in the array under the array argument overwriting the array. |
array:exists(array, condition, [var]) | array, data-manipulation, filter | array | Â |
Filters events based on whether the given array contains an
element that satisfies a given condition (based on the array
argument). Recommended for flat arrays. Does not work on nested
arrays — use objectArray:exists()
instead.
|
array:filter(array, [asArray], function, [var]) | array, data-manipulation, filter | array | Â | Drops entries from the input array using the given filtering function. |
array:length(array, [as]) | array, array-length | array | Â | Counts the number of elements in a flat array. |
array:reduceAll(array, function, var) | aggregate, array, data-manipulation | array | Â | Computes a value from all events and array elements of the specified array. |
array:regex(array, [flags], regex) | array, filter, regular-expression | array | Â | Checks whether the given pattern matches any of the values of the array and excludes the event from the search result. |
array:rename(array, asArray) | array, data-manipulation | array | Â | Takes the name of an array and renames all fields of this array. |
array:sort(array, [asArray], [order], [type]) | array, data-manipulation | array | Â | Sorts the elements of an array of values using the given sorting data type and order. |
asn([as], [field]) | network | field | Â | Determines autonomous system number and organization associated. |
avg([as], field) | aggregate, statistics | field | Â | Calculates the average for a field of a set of events. |
base64Decode([as], [charset], field) | parsing | field | Â | Performs Base64 decoding of a field. |
base64Encode([as], field) | parsing | field | Â | Performs Base64 encoding of a field. |
bitfield:extractFlags(field, [onlyTrue], output) | data-manipulation | Â | Â | Interprets an integer as a bit field and extracts the specified flags. |
bitfield:extractFlagsAsArray([asArray], field, flagNames) | data-manipulation | Â | Â | Interprets an integer as a bit field and extracts the specified flags. Information is returned as an array, listed in ascending order, from lowest bit to highest bit. |
bitfield:extractFlagsAsString([as], field, flagNames, [separator]) | data-manipulation | Â | Â | Interprets an integer as a bit field and extracts the specified flags. Information is returned as a string, listed in ascending order, from lowest bit to highest bit. |
bucket([buckets], [field], [function], [limit], [minSpan], [span], [timezone], [unit]) | aggregate | span | Â |
Extends the groupBy() function for grouping
by time.
|
callFunction([as], field, function) | aggregate | function | Â | Calls the named function on a field over a set of events. |
cidr([column], field, [file], [negate], [subnet]) | filter, network | field | Â | Filters events using CIDR subnets. |
coalesce([as], expressions, [ignoreEmpty]) | data-manipulation | expressions | Â | Selects the value of the first expression from a list of expressions |
collect(fields, [limit], [multival], [separator]) | aggregate | fields | Â | Collects fields from multiple events into one event. |
communityId([as], destinationip, [destinationport], [icmpcode], [icmptype], proto, [seed], sourceip, [sourceport]) | network | Â | Â | Computes the Community ID, a standard for hashing network flows. |
concat([as], field) | data-manipulation, string | field | Â | Concatenates the values of a list of fields into a value in a new field. |
concatArray([as], field, [from], [prefix], [separator], [suffix], [to]) | array, data-manipulation, string | field | Â | Concatenates values of all fields with same name and an array suffix into a new field. |
copyEvent(type) | data-manipulation, event | type | Â | Duplicates an event so that the pipeline will see both events. |
correlate(globalConstraints, includeConstraintValues, includeMatchesOnceOnly, iterationLimit, jitterTolerance, maxPerRoot, query, root, sequence, sequenceBy, within) | aggregate | query | Â | Correlates data by uniting multiple queries |
count([as], [distinct], [field]) | aggregate, statistics | field | Â | Counts given events. |
counterAsRate([as], field) | aggregate | field | Â | Calculates the rate for a counter field. |
createEvents(rawstring) | aggregate-testing, event | rawstring | Â | Generates temporary events as part of the query. |
crypto:md5([as], field) | hash | field | Â | Computes a cryptographic MD5-hashing of an input string. |
crypto:sha1([as], field) | hash | field | Â | Computes a cryptographic SHA1-hashing of an input string. |
crypto:sha256([as], field) | hash | field | Â | Computes a cryptographic SHA256-hashing of an input string. |
default(field, [replaceEmpty], value) | event, format | value | Â | Creates a field to given parameter and given value. |
defineTable([end], include, name, query, [start], [view]) | join, preamble | query | Â | Runs a subquery to generate an in-memory table, then the table results are passed to a primary query to perform join-like operations. |
drop(fields) | data-manipulation, event | fields | Â | Removes specified fields from each event. |
dropEvent() | data-manipulation, event | Â | Â | Drops completely an event in parser pipeline to stop it from being ingested. |
duration([as], duration) | time-date | duration | Â | Computes the number of milliseconds denoted by a human-readable specification. |
end([as]) | time-date | as | Â |
Assign the end of the search time interval to the field provided
by parameter as.
|
eval() | data-manipulation | Â | Â | Creates a new field by evaluating the provided expression. |
eventFieldCount([as]) | event | Â | Â | Computes number of fields event uses internally for the values. |
eventInternals([prefix]) | event | Â | Â | Add a set of fields describing the storage locations of this event. |
eventSize([as]) | event | Â | Â | Determines the number of bytes that this event uses internally for the values, not counting the bytes for storing the field names. |
fieldset() | event | Â | Â | Retrieves a list of available fields. |
fieldstats([limit]) | aggregate, event | Â | Â | Displays statistics about fields. |
findTimestamp([addErrors], [as], [field], [timezone], [timezoneAs], [timezoneField]) | time-date | Â | Â | Finds timestamp in given field and parses, trying multiple timestamp formats. |
format([as], field, format, [timezone]) | data-manipulation, format | format | Â | Formats a string using printf-style. |
formatDuration([as], field, [from], [precision]) | format, time-date | field | Â | Formats a duration into a more readable string. |
formatTime(as, [field], format, [locale], [timezone], [unit]) | format | format | Â |
Formats a string according to strftime().
|
geography:distance([as], lat1, lat2, lon1, lon2) | geolocation | Â | Â | Calculates the distance between two geographical coordinates along an ideal earth surface. |
geohash([as], [lat], [lon], [precision]) | geolocation | Â | Â | Calculates a geohash value given two fields representing latitude and longitude. |
getField([as], source) | data-manipulation | Â | Â | Reads dynamically-named fields that are computed from an expression. |
groupBy(field, [function], [limit]) | aggregate | field | Â | Groups events by specified fields and executes aggregate functions on each group. |
hash([as], field, [limit], [seed]) | event, hash | field | Â | Computes a non-cryptographic hash of a list of fields. |
hashMatch([bits], [field], [hash], input, salt) | security | input | Â | Calculates a secure hash of a field and uses it to match events as a filter. |
hashRewrite([as], [bits], field, [hash], [replaceInRawstring], salt) | security | field | Â | Calculates a secure hash of a field for storing in the event. |
head([limit]) | aggregate | limit | Â | Returns the oldest events from an event stream. |
if([as], condition, else, then) | condition | condition | Â | Computes a value in one of two ways (evaluates one of two expressions). |
in(field, [ignoreCase], values) | comparison, filter | field | Â | Filters records by values where field is in given values. |
ioc:lookup([confidenceThreshold], field, [include], [prefix], [strict], type) | security | field | Â | Look up IOCs (Indicators of Compromise). |
ipLocation([as], [field]) | geolocation, network | field | Â | Determines country, city, longitude, and latitude for given IP address. |
join([end], field, [include], [key], [limit], [live], [max], [mode], query, [repo], [start], [view]) | join | query | Â | Join two LogScale searches. |
json:prettyPrint([as], [field], [step], [strict]) | data-manipulation, format | field | Â | More readable output to a JSON field. |
kvParse([as], [excludeEmpty], [field], [onDuplicate], [override], [prefix], [separator], [separatorPadding]) | parsing | field | Â | Key-value parse events. |
length([as], field) | string | field | Â | Computes the number of characters in a string field. |
linReg([prefix], x, y) | aggregate | Â | Â | Computes linear relationship model between two variables using least-squares fitting. |
lower([as], field, [locale], [type]) | format | field | Â | Changes text of a given string field to lower case letters. |
lowercase(field, [include], [locale]) | data-manipulation, format, string | field | Â | Changes field name or content to lowercase for parsers. |
match([column], field, file, [glob], [ignoreCase], [include], [mode], [nrows], [strict]) | filter | file | Â | Searches text using a CSV or JSON file and can enhance entries. |
math:abs([as], field) | math | field | Â | Calculates the absolute value of a field; the result is always a positive number or 0. |
math:arccos([as], field) | math | field | Â | Calculates the arc cosine of a field. |
math:arcsin([as], field) | math | field | Â | Calculates the arc sine of a field. |
math:arctan([as], field) | math | field | Â | Calculates the arc tangent of a value. |
math:arctan2([as], x, y) | math | Â | Â | Calculates the arc tangent of a value. |
math:ceil([as], field) | math | field | Â | Rounds field value to smallest integer that's larger than or equal to it. |
math:cos([as], field) | math | field | Â | Calculates the cosine of a field. |
math:cosh([as], field) | math | Â | Â | Computes the hyperbolic cosine of a double field. |
math:deg2rad([as], field) | math | field | Â | Converts angles from degrees to radians. |
math:exp([as], field) | math | field | Â | Calculates Euler's number e raised to the power of a double value in a field. |
math:expm1([as], field) | math | field | Â | Calculates the exponential value of a number minus 1. |
math:floor([as], field) | math | field | Â | Computes the largest integer value not greater than the field value given. |
math:log([as], field) | math | field | Â | Calculates the natural logarithm (base e) of the value in a double field. |
math:log10([as], field) | math | field | Â | Calculates the base 10 logarithm of a double field. |
math:log1p([as], field) | math | field | Â | Calculates the natural logarithm of the sum of field's value and 1. |
math:log2([as], field) | math | field | Â | Calculates the base 2 logarithm of a double field. |
math:mod([as], divisor, field) | math | field | Â | Calculates the floor modulus of field value and the divisor. |
math:pow([as], exponent, field) | math | field | Â | Calculates the field value to the exponent power. |
math:rad2deg([as], field) | math | field | Â | Converts angles from radians to degrees. |
math:sin([as], field) | math | field | Â | Calculates the sine of a field. |
math:sinh([as], field) | math | field | Â | Calculates the hyperbolic sine of a double field. |
math:sqrt([as], field) | math | field | Â | Calculates the rounded positive square root of a double field. |
math:tan([as], field) | math | field | Â | Calculates the trigonometric tangent of an angle in a field. |
math:tanh([as], field) | math | field | Â | Calculates the hyperbolic tangent of a field. |
max([as], field) | aggregate, statistics | field | Â | Finds the largest number for the specified field over a set of events. |
min([as], field) | aggregate, statistics | field | Â | Finds the smallest number for the specified field over a set of events. |
neighbor([direction], [distance], include, [prefix]) | aggregate, data-manipulation, sequence | include | Â | Allows access to fields from a single neighboring event in a sequence. |
now([as]) | time-date | as | Â |
Assign the current time to the field provided by parameter
as.
|
objectArray:eval(array, asArray, function, [var]) | array, nested-array | array | Â | Maps over an array of objects and outputs a new array of the mapped values. |
objectArray:exists(array, condition, [var]) | array, nested array | array | Â | Filters events based on whether the given array contains an element that satisfies a given condition (based on the array argument). Recommended for nested arrays. |
parseCEF([field], [headerprefix], [keeplabels], [labelprefix], [prefix]) | data-manipulation, parsing | field | Â | Parses CEF version 0.x encoded messages. |
parseCsv(columns, [delimiter], [excludeEmpty], field, [trim]) | data-manipulation, parsing | field | Â | Parses a CSV-encoded field into known columns. |
parseFixedWidth(columns, [field], [trim], widths) | parsing | field | Â | Parses a fixed width-encoded field into known columns. |
parseHexString([as], [charset], field) | data-manipulation, math, parsing | field | Â | Parses input from hex encoded bytes, decoding resulting bytes as a string. |
parseInt([as], [endian], field, [radix]) | math, parsing | field | Â | Converts an integer from any radix or base to base-ten, decimal radix. |
parseJson([exclude], [excludeEmpty], field, [handleNull], [include], [prefix], [removePrefixes]) | data-manipulation, parsing | field | Â | Parses specified fields as JSON. |
parseLEEF([delimiter], [field], [headerprefix], [keeplabels], [labelprefix], [parsetime], [prefix], [timezone]) | data-manipulation, parsing | field | Â | Parses LEEF version 1.0 and 2.0 encoded messages. |
parseTimestamp([addErrors], [as], [caseSensitive], field, [format], [timezone], [timezoneAs], [timezoneField]) | parsing, time-date | format | Â | Parses a string into a timestamp. |
parseUri([defaultBase], field, [prefix]) | parsing | field | Â | Extracts URI components from a field. |
parseUrl([as], [field]) | parsing | field | Â | Extracts URL components from a field. |
parseXml(field, [prefix], [strict]) | parsing | field | Â | Parses specified field as XML. |
partition(condition, function, [split]) | aggregate, data-manipulation, sequence | function | Â | Splits a sequence of events into multiple partitions based on a condition. |
percentage([as], condition) | aggregate | condition | Â | Calculates what portion of events meet specified conditions. |
percentile([accuracy], [as], field, [percentiles]) | aggregate, statistics | field | Â | Finds one event with a field for each percentile specified. |
range([as], field) | aggregate, statistics | field | Â | Finds numeric range between smallest and largest numbers for field over a set of events. |
rdns([as], field, [limit], [server]) | aggregate, network | field | Â | Events using RDNS lookup. |
readFile(file, [include], [limit]) | data-manipulation, event | file | Â |
Uses a .csv lookup file or ad-hoc table as
data input for the query.
|
regex([field], [flags], [limit], regex, [repeat], [strict]) | filter, regular-expression, string | regex | Â | Extracts new fields using a regular expression. |
rename([as], field) | data-manipulation, event | field | Â | Renames one or more given fields. |
replace([as], [field], [flags], regex, [replacement], [with]) | data-manipulation, regular-expression, string | regex | Â | Replaces each substring that matches given regular expression with given replacement. |
reverseDns([as], field, [limit], [server]) | transformation | field | Â | Performs reverse DNS lookups. |
round([as], field, [how]) | math | field | Â | Rounds an input field up or down, depending on which is nearest. |
sample([field], [percentage]) | filter, statistics | percentage | Â | Samples the event stream. |
sankey(source, target, [weight]) | aggregate, data-manipulation, widget | Â | Â | Produces data compatible with Sankey widget. |
select(fields) | event | fields | Â | Used to specify a set of fields to select from each event. |
selectFromMax(field, include) | aggregate | field | Â | Selects event with the largest value for the specified field. |
selectFromMin(field, include) | aggregate | field | Â | Selects event with the smallest value for the specified field. |
selectLast(fields) | aggregate | fields | Â | Specify fields to select from events, keeping value of most recent event for each field. |
selfJoin([collect], field, [limit], [postfilter], [prefilter], [select], where) | join | field | Â | Used to collate data from events that share a key. |
selfJoinFilter(field, [prefilter], where) | filter, join | field | Â | Runs query to determine IDs, and then gets all events containing one of them. |
series(collect, [endmatch], [maxduration], [maxpause], [memlimit], [separator], [startmatch]) | aggregate, data-manipulation | collect | Â | Collects a series of values for selected fields from multiple events into one or more events. |
session([function], [maxpause]) | aggregate, statistics | function | Â | Collects events into sessions, and aggregates them. |
setField(target, value) | data-manipulation | Â | Â | Sets fields whose names are not known but computed from an expression. |
setTimeInterval([end], start, [timezone]) | preamble, time-date | start | Â | Sets the query time interval (overriding the setting in the API or UI) |
shannonEntropy([as], field) | network | field | Â | Calculates a entropy measure from a string of characters. |
slidingTimeWindow([current], function, span, [timestampfield]) | aggregate, data-manipulation, sequence | function | Â | Applies an aggregation to a moving time-based window of events in a sequence. |
slidingWindow([current], events, function) | aggregate, data-manipulation, sequence | function | Â | Applies an aggregation to a moving window of a specified number of events in a sequence. |
sort([field], [limit], [order], [reverse], [type]) | aggregate | field | Â | Sorts events by their fields. |
split([field], [strip]) | array, data-manipulation | field | Â | Splits an event structure created by a JSON array into distinct events. |
splitString([as], by, [field], [index]) | array, data-manipulation, regular-expression, string | field | Â | Splits a string by specifying a regular expression by which to split. |
start([as]) | time-date | as | Â |
Assign the start of the search time interval to the field provided
by parameter as.
|
stats([function]) | aggregate, statistics | function | Â | Used to compute multiple aggregate functions over the input. |
stdDev([as], field) | aggregate, statistics | field | Â | Calculates the standard deviation for a field over a set of events. |
stripAnsiCodes([as], field) | data-manipulation, string | field | Â | Removes ANSI color codes and movement commands. |
subnet([as], bits, field) | network | field | Â | Computes a subnet from a IPV4 field. |
sum([as], field) | aggregate | field | Â | Calculates the sum for a field over a set of events. |
table(fields, [limit], [order], [reverse], [sortby], [type]) | aggregate, widget | fields | Â | Used to create a widget to present the data in a table. |
tail([limit]) | aggregate | limit | Â | Returns the newest events from an event stream. |
test(expression) | comparison, event, filter | expression | Â | Evaluates boolean expression and filters events. |
text:contains(string, substring) | data-manipulation, filter | string | Â | Tests if a specific substring is present within a given string. |
text:endsWith(string, substring) | data-manipulation, filter | string | Â | Tests if a specific substring is present at the end of a given string. |
text:length([as], string) | data-manipulation, string | string | added in 1.207 | Computes the length of a string. |
text:positionOf([as], [begin], character, [occurrence], string) | data-manipulation, string | string | added in 1.207 | Computes the position of a given character or substring within a string. |
text:startsWith(string, substring) | data-manipulation, filter | string | Â | Tests if a specific substring is present at the start of a given string. |
text:substring([as], [begin], [end], string) | data-manipulation, string | string | added in 1.207 | Extracts a substring from a string given a pair of positions into the string. |
time:dayOfMonth([as], [field], [timezone], [timezoneField]) | time-date | field | Â | Gets the day of the month of a timestamp field. |
time:dayOfWeek([as], field, [timezone], [timezoneField]) | time-date | field | Â | Gets day of week from 1 (Mon) to 7 (Sun) of a timestamp. |
time:dayOfWeekName([as], [field], [timezone], [timezoneField]) | time-date | field | Â | Gets the English display name of day of the week of a timestamp field. |
time:dayOfYear([as], [field], [timezone], [timezoneField]) | time-date | field | Â | Gets the day of the year of a timestamp field, from 1 to 365, or 366 in a leap year. |
time:hour([as], [field], [timezone], [timezoneField]) | time-date | field | Â | Gets the hour (24-hour clock) of a timestamp field. |
time:millisecond([as], [field], [timezone], [timezoneField]) | time-date | field | Â | Gets the millisecond of a timestamp field. |
time:minute([as], [field], [timezone], [timezoneField]) | time-date | field | Â | Gets the minute value of a timestamp field. |
time:month([as], [field], [timezone], [timezoneField]) | time-date | field | Â | Gets the month of a timestamp field (from 1 to 12). |
time:monthName([as], [field], [timezone], [timezoneField]) | time-date | field | Â | Gets the English name of month of a timestamp field (for example, January). |
time:second([as], [field], [timezone], [timezoneField]) | time-date | field | Â | Gets the second of a timestamp field. |
time:weekOfYear([as], [field], [timezone], [timezoneField]) | time-date | field | Â | Gets the week number within a year of a timestamp, a value from 1 to 53. |
time:year([as], [field], [timezone], [timezoneField]) | time-date | field | Â | Gets the year of a timestamp field. |
timeChart([buckets], [function], [limit], [minSpan], [series], [span], [timezone], [unit]) | aggregate, widget | series | Â | Used to draw a linechart where the x-axis is time. |
tokenHash([as], field) | hash, string | field | Â | Calculates a hash by tokenizing the input string (split by spaces), creating a hash for each token and then added the result together. This generates the same hash value, even if the order of the individual values in the source string is different. |
top([as], [error], field, [limit], [max], [percent], [rest], [sum]) | aggregate | field | Â | Finds the top results based on a given field. |
transpose([column], [header], [limit], [pivot]) | aggregate, data-manipulation | pivot | Â | Transposes a query results set by creating an event for each attribute. |
unit:convert([as], [binary], field, [from], [keepUnit], [to], [unit]) | data-manipulation, math | field | Â | Converts values between different units. |
upper([as], field, [locale]) | data-manipulation, format | field | Â | Changes contents of a string field to upper case letters. |
urlDecode([as], field) | data-manipulation, format, network | field | Â | URL-decodes the contents of a string field. |
urlEncode([as], field, [type]) | data-manipulation, format, network | field | Â | URL encodes the contents of a string field. |
wildcard([field], [ignoreCase], [includeEverythingOnAsterisk], pattern) | filter | pattern | Â | Performs a wildcard pattern search with optional case insensitivity. |
window([buckets], [function], [span]) | aggregate | function | Â | Computes aggregate functions over a sliding window of data. |
worldMap([ip], [lat], [lon], [magnitude], [precision]) | aggregate, widget | Â | Â | Used to produce data compatible with the World Map widget. |
writeJson([as], [field]) | data-manipulation, format | field | Â | Writes data, including fields, as a JSON object. |
xml:prettyPrint([as], field, [step], [strict], [width]) | data-manipulation | field | Â | Nicer output to an XML field. |