Query Functions

Manipulating and formatting functions for extracting information from your event data

LogScale's query functions take a set of events, parameters, or configurations. From this, they produce, reduce, or modify values within that set, or in the events themselves within a query pipeline.

Functions broadly fall into two categories, Transformation and Aggregate:

  • Transformation functions, sometimes referred to as Filter functions, transform or filter data and may add, remove or modify fields.

  • Aggregate functions combine events into a new results — often a single number or row.

Below is an alphabetical listing of all of LogScale query functions.

Table: Query Functions

FunctionImplied ArgumentAvailabilityDescription
array:contains(array, value)arrayintroduced in 1.62Checks whether the given value matches any of the values of the array and excludes the event if no value matches
array:regex(array, [flags], regex)arrayintroduced in 1.62Checks whether the given pattern matches any of the values of the array and excludes the event from the search result
asn([as], [field])field Determines autonomous system number and organization associated.
avg([as], field)field Calculates the average for a field of a set of events.
base64Decode([as], [charset], field)field Performs Base64 decoding of a field.
beta:param(as, parameter)parameter Reads given parameter and assigns the value to a field in the event.
beta:repeating([as], [interval])interval Marks the live query the function is used in as repeating.
bucket([buckets], [field], [function], [limit], [minSpan], [span], [timezone], [unit])span Extends the groupBy function for grouping by time.
callFunction([as], field, function)function Calls the named function on a field over a set of events.
cidr([column], [field], [file], [negate], [subnet])field Filters events using CIDR subnets.
collect(fields, [limit], [multival], [separator])fields Collects fields from multiple events into one event.
communityId([as], destinationip, [destinationport], [icmpcode], [icmptype], proto, [seed], sourceip, [sourceport]) introduced in 1.33Computes the Community ID, a standard for hashing network flows.
concat([as], field)field Concatenates the values of a list of fields into a value in a new field.
concatArray([as], field, [from], [prefix], [separator], [suffix], [to])field Concatenates values of all fields with same name and an array suffix into a new field.
copyEvent(type)type Duplicates event so pipeline will see both events.
count([as], [distinct], [field])field Counts given events.
counterAsRate([as], field)field Calculates the rate for a counter field.
createEvents([rawstring])  Generates temporary events as part of the query.
default(field, [replaceEmpty], value)value Creates a field to given parameter and given value.
drop(fields)fields Removes attributes or columns from a result set.
dropEvent()  Drops completely an event in parser pipeline to stop it from being ingested.
end([as])as Assign the end of the search time interval to the field provided by parameter as.
eval()  Creates a new field by evaluating the provided expression.
eventFieldCount([as])  Returns number of fields event uses internally for the values.
eventInternals([prefix])  Add a set of fields describing the storage locations of this event.
eventSize([as])  Returns the number of bytes that this event uses internally for the values, not counting the bytes for storing the field names.
fieldset()  Retrieves a list of available fields.
fieldstats()  Retrieves stats about fields.
findTimestamp([addErrors], [as], [field], [timezone], [timezoneAs])  Finds timestamp in given field and parses, trying multiple timestamp formats.
format([as], field, format, [timezone])format Formats a string using printf-style.
formatDuration([as], field, [from], [precision])field Formats a duration into a more readable string.
formatTime(as, [field], format, [locale], [timezone])format Formats a string according to strftime, similar to unix strftime.
geography:distance([as], lat1, lat2, lon1, lon2)  Calculates the distance between two geographical coordinates along an ideal earth surface
geohash([as], [lat], [lon], [precision])geohash Calculates a geohash value given two fields representing latitude and longitude.
groupBy(field, [function], [limit])field Groups events by specified fields and executes aggregate functions on each group.
hash([as], field, [limit], [seed])field Computes a non-cryptographic hash of a list of fields.
hashMatch([bits], [field], [hash], input, [salt])input Calculates a secure hash of a field and uses it to match events as a filter.
hashRewrite([as], [bits], field, [hash], [replaceInRawstring], salt)field Calculates a secure hash of a field for storing in the event.
head([limit])limit Returns the oldest events.
holtwinters([alpha], [beta], [gamma], period)perioddeprecated in 1.73Used to generate a trendline for a periodic dataset.
in(field, values)field Filters records by values where field is in given values.
ioc:lookup([confidenceThreshold], field, [include], [prefix], [strict], type)field Look up IOCs (indicators of compromise).
ipLocation([as], [field])field Determines country, city, longitude, and latitude for given IP address.
join([end], field, [include], [key], [limit], [live], [max], [mode], [query], [repo], [start], [view])query Join two Humio searches.
json:prettyPrint([as], [field], [step], [strict])field Nicer output to a JSON field.
kvParse([as], [excludeEmpty], [field], [override], [separator], [separatorPadding])field Key-value parse events.
length([as], field)field Returns the number of characters in a string field.
linReg([prefix], x, y)linreg Computes linear relationship model between two variables using least-squares fitting.
lookup(from, [include], on)fromdeprecated in 1.30.0Enhances events with metadata.
lower([as], field, [type])field Changes text of a given string field to lower-case letters.
lowercase(field, [include], [locale])field Changes field name or content to lowercase for parsers.
match([column], field, file, [glob], [ignoreCase], [include], [mode], [strict])file Searches text using a CSV or JSON file and can enhance entries.
math:abs([as], field)field Calculates the absolute value of a field.
math:arccos([as], field)field Calculates the arc cosine of a field.
math:arcsin([as], field)field Calculates the arc sine of a field.
math:arctan([as], field)field Calculates the arc tangent of a value.
math:arctan2([as], xfield, yfield)  Calculates the arc tangent of a value.
math:ceil([as], field)field Rounds field value to smallest integer that's larger than or equal to it.
math:cos([as], field)field Calculates the cosine of a field.
math:cosh([as], field)  Returns the hyperbolic cosine of a double field.
math:deg2rad([as], field)field Converts angles from degrees to radians.
math:exp([as], field)field Calculates Euler's number e raised to the power of a double value in a field.
math:expm1([as], field)field Returns the exponential value of a number minus 1.
math:floor([as], field)field Returns largest integer value not greater than the field value given.
math:log([as], field)field Calculates the natural logarithm (base e) of the value in a double field.
math:log10([as], field)field Calculates the base 10 logarithm of a double field.
math:log1p([as], field)field Calculates the natural logarithm of the sum of field's value and 1.
math:log2([as], field)field Calculates the base 2 logarithm of a double field.
math:mod([as], divisor, field)field Calculates the floor modulus of field value and the divisor.
math:pow([as], exponent, field)field Calculates the field value to the exponent power.
math:rad2deg([as], field)field Converts angles from radians to degrees.
math:sin([as], field)field Calculates the sine of a field.
math:sinh([as], field)field Calculates the hyperbolic sine of a double field.
math:spherical2cartesian([as], azimuth, polar, radius)  Calculates the average for a field of a set of events.
math:sqrt([as], field)field Calculates the rounded positive square root of a double field.
math:tan([as], field)field Calculates the trigonometric tangent of an angle in a field.
math:tanh([as], field)field Calculates the hyperbolic tangent of a field.
max([as], field, [type])field Finds the largest number for the specified field over a set of events.
min([as], field, [type])field Finds the smallest number for the specified field over a set of events.
now([as])as Assign the current time to the field provided by parameter as.
parseCEF([field], [prefix])field Parses CEF version 0.x encoded messages.
parseCsv(columns, [delimiter], [excludeEmpty], field)field Parses a CSV-encoded field into known columns.
parseFixedWidth(columns, [field], [trim], widths)field Parses a fixed width-encoded field into known columns.
parseHexString([as], [charset], field)field Parses input from hex encoded bytes, decoding resulting bytes as a string.
parseInt([as], [endian], field, [radix])field Converts an integer from any radix or base to base-ten, decimal radix.
parseJson([exclude], field, [include], [prefix], [removePrefixes])field Parses specified fields as JSON.
parseLEEF([delimiter], [field], [parsetime], [prefix], [timezone])field Parses LEEF version 1.0 and 2.0 encoded messages.
parseTimestamp([addErrors], [as], [caseSensitive], field, [format], [timezone], [timezoneAs])  Parses a string into a timestamp.
parseUrl([as], [field])field Extracts URL components from a field.
parseXml(field, [prefix], [strict])field Parses specified field as XML.
percentile([accuracy], [as], field, [percentiles])field Returns one event with a field for each percentile specified.
range([as], field)field Finds numeric range between smallest and largest numbers for field over a set of events.
rdns([as], field, [server])field Events using RDNS lookup.
regex([field], [flags], [limit], regex, [repeat], [strict])regex Extracts new fields using a regular expression.
rename(as, field)field Renames a given field.
replace([as], [field], [flags], regex, [replacement], [with])field Replaces each substring that matches given regular expression with given replacement.
round([as], field, [how])field Rounds an input field up or down, depending on which is nearest.
sample([field], [percentage])percentage Samples the event stream.
sankey(source, target, [weight])sankey Produces data compatible with Sankey widget.
select(fields)fields Used to specify a set of fields to select from each event.
selectFromMax(field, include)  Selects event with the largest value for the specified field.
selectFromMin(field, include)  Selects event with the smallest value for the specified field.
selectLast(fields)fields Specify fields to select from events, keeping value of most recent event for each field.
selfJoin([collect], field, [limit], [postfilter], [prefilter], [select], where)field Used to collate data from events that share a key.
selfJoinFilter(field, [prefilter], where)field Runs query to determine IDs, and then gets all events containing one of them.
series(collect, [endmatch], [maxduration], [maxpause], [memlimit], [separator], [startmatch])collect Collects a series of values for selected fields from multiple events into one or more events.
session([function], [maxpause])function Collects events into sessions, and aggregates them.
shannonEntropy([as], field)field Calculates a entropy measure from a string of characters.
sort([field], [limit], [order], [reverse], [type])field Sorts events by their fields.
split([field], [strip])field Splits an event structure created by a JSON array into distinct events.
splitString([as], by, [field], [index])field Splits a string by specifying a regular expression by which to split.
start([as])as Assign the start of the search time interval to the field provided by parameter as.
stats([function])function Used to compute multiple aggregate functions over the input.
stdDev([as], field)field Calculates the standard deviation for a field over a set of events.
stripAnsiCodes([as], field, [type])field Removes ANSI color codes and movement commands.
subnet([as], bits, field)field Computes a subnet from a IPV4 field.
sum([as], field)field Calculates the sum for a field over a set of events.
table(fields, [limit], [order], [reverse], [sortby], [type])fields Used to create a widget to present the data in a table.
tail([limit])limit Returns the newest events.
test(expression)expression Evaluates boolean expression and filters events.
time:dayOfMonth([as], [field], [timezone], [timezoneField])field Gets the day of the month of a timestamp field.
time:dayOfWeek([as], field, [timezone], [timezoneField])field Gets day of week from 1 (Mon) to 7 (Sun) of a timestamp.
time:dayOfWeekName([as], [field], [timezone], [timezoneField])field Gets the English display name of day of the week of a timestamp field.
time:dayOfYear([as], [field], [timezone], [timezoneField])field Gets the day of the year of a timestamp field, from 1 to 365, or 366 in a leap year.
time:hour([as], [field], [timezone], [timezoneField])field Gets the hour (24-hour clock) of a timestamp field.
time:millisecond([as], [field], [timezone], [timezoneField])field Gets the millisecond of a timestamp field.
time:minute([as], [field], [timezone], [timezoneField])field Gets the minute value of a timestamp field.
time:month([as], [field], [timezone], [timezoneField])field Gets the month of a timestamp field (from 1 to 12).
time:monthName([as], [field], [timezone], [timezoneField])field Gets the English name of month of a timestamp field (e.g., January).
time:second([as], [field], [timezone], [timezoneField])field Gets the second of a timestamp field.
time:weekOfYear([as], [field], [timezone], [timezoneField])field Gets the week number within a year of a timestamp, a value from 1 to 53.
time:year([as], [field], [timezone], [timezoneField])field Gets the year of a timestamp field.
timeChart([buckets], [function], [limit], [minSpan], [series], [span], [timezone], [unit])series Used to draw a linechart where the x-axis is time.
tokenHash([as], field)field Calculates a structure hash which is equal for similarly structured input.
top([as], [error], field, [limit], [max], [percent], [rest], [sum])field Finds the top results based on a given field.
transpose([column], [header], [limit], [pivot])pivot Transposes a query results set by creating an event for each attribute.
unit:convert([as], [binary], field, [from], [keepUnit], [to], [unit])field Converts values between different units.
upper([as], field, [locale])field Changes contents of a string field to upper-case letters.
urlDecode([as], field)field URL-decodes the contents of a string field.
urlEncode([as], field, [type])field URL-encodes the contents of a string field.
window([buckets], [function], [span])function Computes aggregate functions over a sliding window of data.
worldMap([ip], [lat], [lon], [magnitude], [precision])worldmap Used to produce data compatible with the World Map widget.
writeJson([as], [field])field Writes data, including fields, as a JSON object.
xml:prettyPrint([as], field, [step], [strict], [width])field Nicer output to an XML field.