Query Functions

Manipulating and formatting functions for extracting information from your event data

LogScale's query functions take a set of events, parameters, or configurations. From this, they produce, reduce, or modify values within that set, or in the events themselves within a query pipeline.

Functions broadly fall into two categories, Transformation and Aggregate:

  • Transformation functions, sometimes referred to as Filter functions, transform or filter data and may add, remove or modify fields.

  • Aggregate functions combine events into a new results — often a single number or row.

Below is an alphabetical listing of all of LogScale query functions.

Table: Query Functions

FunctionTypeDefault ArgumentAvailabilityDescription
array:contains(array, value)array, data-manipulation, filterarrayintroduced in 1.62Checks whether the given value matches any of the values of the array and excludes the event if no value matches
array:filter(array, [asArray], function, var)data-manipulation, arrayarrayintroduced in 1.39Drops entries from the input array using the given filtering function.
array:reduceAll(array, function, var)data-manipulation, arrayarrayintroduced in 1.77Computes a value from all events and array elements of the specified array.
array:regex(array, [flags], regex)regular-expression, array, filterarrayintroduced in 1.62Checks whether the given pattern matches any of the values of the array and excludes the event from the search result
asn([as], [field])networkfield Determines autonomous system number and organization associated.
avg([as], field)aggregatefield Calculates the average for a field of a set of events.
base64Decode([as], [charset], field)mathfield Performs Base64 decoding of a field.
beta:param(as, parameter)data-manipulationparameter Reads given parameter and assigns the value to a field in the event.
beta:repeating([as], [interval])data-manipulationinterval Marks the live query the function is used in as repeating.
bitfield:extractFlags(field, output)data-manipulation introduced in 1.76Interprets an integer as a bit field and extracts the specified flags
bucket([buckets], [field], [function], [limit], [minSpan], [span], [timezone], [unit])aggregatespan Extends the groupBy function for grouping by time.
callFunction([as], field, function)aggregatefunction Calls the named function on a field over a set of events.
cidr([column], [field], [file], [negate], [subnet])network, filterfield Filters events using CIDR subnets.
collect(fields, [limit], [multival], [separator])aggregatefields Collects fields from multiple events into one event.
communityId([as], destinationip, [destinationport], [icmpcode], [icmptype], proto, [seed], sourceip, [sourceport])network introduced in 1.33Computes the Community ID, a standard for hashing network flows.
concat([as], field)string, data-manipulationfield Concatenates the values of a list of fields into a value in a new field.
concatArray([as], field, [from], [prefix], [separator], [suffix], [to])string, data-manipulationfield Concatenates values of all fields with same name and an array suffix into a new field.
copyEvent(type)eventtype Duplicates event so pipeline will see both events.
count([as], [distinct], [field])statistics, aggregatefield Counts given events.
counterAsRate([as], field)aggregatefield Calculates the rate for a counter field.
createEvents([rawstring])event, aggregate-testing  Generates temporary events as part of the query.
default(field, [replaceEmpty], value)eventvalue Creates a field to given parameter and given value.
drop(fields)eventfields Removes attributes or columns from a result set.
dropEvent()event  Drops completely an event in parser pipeline to stop it from being ingested.
end([as])time-dateas Assign the end of the search time interval to the field provided by parameter as.
eval()data-manipulation  Creates a new field by evaluating the provided expression.
eventFieldCount([as])event  Returns number of fields event uses internally for the values.
eventInternals([prefix])event  Add a set of fields describing the storage locations of this event.
eventSize([as])event  Returns the number of bytes that this event uses internally for the values, not counting the bytes for storing the field names.
fieldset()event  Retrieves a list of available fields.
fieldstats()event  Retrieves stats about fields.
findTimestamp([addErrors], [as], [field], [timezone], [timezoneAs])time-date  Finds timestamp in given field and parses, trying multiple timestamp formats.
format([as], field, format, [timezone])formatting, data-manipulationformat Formats a string using printf-style.
formatDuration([as], field, [from], [precision])formattingfield Formats a duration into a more readable string.
formatTime(as, [field], format, [locale], [timezone])formattingformat Formats a string according to strftime().
geography:distance([as], lat1, lat2, lon1, lon2)geolocation  Calculates the distance between two geographical coordinates along an ideal earth surface
geohash([as], [lat], [lon], [precision])geolocationgeohash Calculates a geohash value given two fields representing latitude and longitude.
groupBy(field, [function], [limit])aggregatefield Groups events by specified fields and executes aggregate functions on each group.
hash([as], field, [limit], [seed])eventfield Computes a non-cryptographic hash of a list of fields.
hashMatch([bits], [field], [hash], input, [salt])comparison, filterinput Calculates a secure hash of a field and uses it to match events as a filter.
hashRewrite([as], [bits], field, [hash], [replaceInRawstring], salt)comparison, data-manipulationfield Calculates a secure hash of a field for storing in the event.
head([limit])aggregatelimit Returns the oldest events.
holtwinters([alpha], [beta], [gamma], period)aggregateperiodremoved in 1.73Used to generate a trendline for a periodic dataset.
in(field, values)comparison, filterfield Filters records by values where field is in given values.
ioc:lookup([confidenceThreshold], field, [include], [prefix], [strict], type)geolocationfield Look up IOCs (indicators of compromise).
ipLocation([as], [field])geolocationfield Determines country, city, longitude, and latitude for given IP address.
join([end], field, [include], [key], [limit], [live], [max], [mode], [query], [repo], [start], [view])joinquery Join two LogScale searches.
json:prettyPrint([as], [field], [step], [strict])formatting, data-manipulationfield Nicer output to a JSON field.
kvParse([as], [excludeEmpty], [field], [override], [separator], [separatorPadding])parsingfield Key-value parse events.
length([as], field)stringfield Returns the number of characters in a string field.
linReg([prefix], x, y)aggregatelinreg Computes linear relationship model between two variables using least-squares fitting.
lookup(from, [include], on)formatfromdeprecated in 1.30.0Enhances events with metadata.
lower([as], field, [type])formattingfield Changes text of a given string field to lower-case letters.
lowercase(field, [include], [locale])string, data-manipulation, formattingfield Changes field name or content to lowercase for parsers.
match([column], field, file, [glob], [ignoreCase], [include], [mode], [strict])stringfile Searches text using a CSV or JSON file and can enhance entries.
math:abs([as], field)mathfield Calculates the absolute value of a field.
math:arccos([as], field)mathfield Calculates the arc cosine of a field.
math:arcsin([as], field)mathfield Calculates the arc sine of a field.
math:arctan([as], field)mathfield Calculates the arc tangent of a value.
math:arctan2([as], xfield, yfield)math  Calculates the arc tangent of a value.
math:ceil([as], field)mathfield Rounds field value to smallest integer that's larger than or equal to it.
math:cos([as], field)mathfield Calculates the cosine of a field.
math:cosh([as], field)math  Returns the hyperbolic cosine of a double field.
math:deg2rad([as], field)mathfield Converts angles from degrees to radians.
math:exp([as], field)mathfield Calculates Euler's number e raised to the power of a double value in a field.
math:expm1([as], field)mathfield Returns the exponential value of a number minus 1.
math:floor([as], field)mathfield Returns largest integer value not greater than the field value given.
math:log([as], field)mathfield Calculates the natural logarithm (base e) of the value in a double field.
math:log10([as], field)mathfield Calculates the base 10 logarithm of a double field.
math:log1p([as], field)mathfield Calculates the natural logarithm of the sum of field's value and 1.
math:log2([as], field)mathfield Calculates the base 2 logarithm of a double field.
math:mod([as], divisor, field)mathfield Calculates the floor modulus of field value and the divisor.
math:pow([as], exponent, field)mathfield Calculates the field value to the exponent power.
math:rad2deg([as], field)mathfield Converts angles from radians to degrees.
math:sin([as], field)mathfield Calculates the sine of a field.
math:sinh([as], field)mathfield Calculates the hyperbolic sine of a double field.
math:spherical2cartesian([as], azimuth, polar, radius)math  Calculates the average for a field of a set of events.
math:sqrt([as], field)mathfield Calculates the rounded positive square root of a double field.
math:tan([as], field)mathfield Calculates the trigonometric tangent of an angle in a field.
math:tanh([as], field)mathfield Calculates the hyperbolic tangent of a field.
max([as], field, [type])statisticsfield Finds the largest number for the specified field over a set of events.
min([as], field, [type])statisticsfield Finds the smallest number for the specified field over a set of events.
now([as])time-dateas Assign the current time to the field provided by parameter as.
parseCEF([field], [prefix])data-manipulation, parsingfield Parses CEF version 0.x encoded messages.
parseCsv(columns, [delimiter], [excludeEmpty], field)data-manipulation, parsingfield Parses a CSV-encoded field into known columns.
parseFixedWidth(columns, [field], [trim], widths)parsingfield Parses a fixed width-encoded field into known columns.
parseHexString([as], [charset], field)data-manipulation, parsingfield Parses input from hex encoded bytes, decoding resulting bytes as a string.
parseInt([as], [endian], field, [radix])parsingfield Converts an integer from any radix or base to base-ten, decimal radix.
parseJson([exclude], [excludeEmpty], field, [handleNull], [include], [prefix], [removePrefixes])parsing, data-manipulationfield Parses specified fields as JSON.
parseLEEF([delimiter], [field], [parsetime], [prefix], [timezone])data-manipulation, parsingfield Parses LEEF version 1.0 and 2.0 encoded messages.
parseTimestamp([addErrors], [as], [caseSensitive], field, [format], [timezone], [timezoneAs])parsing, time-date  Parses a string into a timestamp.
parseUrl([as], [field])parsingfield Extracts URL components from a field.
parseXml(field, [prefix], [strict])parsingfield Parses specified field as XML.
percentile([accuracy], [as], field, [percentiles])statisticsfield Returns one event with a field for each percentile specified.
range([as], field)statisticsfield Finds numeric range between smallest and largest numbers for field over a set of events.
rdns([as], field, [server])networkfield Events using RDNS lookup.
regex([field], [flags], [limit], regex, [repeat], [strict])regular-expression, stringregex Extracts new fields using a regular expression.
rename(as, field)data-manipulation, eventfield Renames a given field.
replace([as], [field], [flags], regex, [replacement], [with])string, data-manipulation, regular-expressionregex Replaces each substring that matches given regular expression with given replacement.
round([as], field, [how])mathfield Rounds an input field up or down, depending on which is nearest.
sample([field], [percentage])statistics, filterpercentage Samples the event stream.
sankey(source, target, [weight])data-manipulation, widgetsankey Produces data compatible with Sankey widget.
select(fields)eventfields Used to specify a set of fields to select from each event.
selectFromMax(field, include)aggregate  Selects event with the largest value for the specified field.
selectFromMin(field, include)aggregate  Selects event with the smallest value for the specified field.
selectLast(fields)aggregatefields Specify fields to select from events, keeping value of most recent event for each field.
selfJoin([collect], field, [limit], [postfilter], [prefilter], [select], where)joinfield Used to collate data from events that share a key.
selfJoinFilter(field, [prefilter], where)filter, joinfield Runs query to determine IDs, and then gets all events containing one of them.
series(collect, [endmatch], [maxduration], [maxpause], [memlimit], [separator], [startmatch])aggregate, data-manipulationcollect Collects a series of values for selected fields from multiple events into one or more events.
session([function], [maxpause])statisticsfunction Collects events into sessions, and aggregates them.
shannonEntropy([as], field)networkfield Calculates a entropy measure from a string of characters.
sort([field], [limit], [order], [reverse], [type])aggregatefield Sorts events by their fields.
split([field], [strip])data-manipulation, string, regular-expressionfield Splits an event structure created by a JSON array into distinct events.
splitString([as], by, [field], [index])data-manipulation, stringfield Splits a string by specifying a regular expression by which to split.
start([as])time-dateas Assign the start of the search time interval to the field provided by parameter as.
stats([function])statisticsfunction Used to compute multiple aggregate functions over the input.
stdDev([as], field)statisticsfield Calculates the standard deviation for a field over a set of events.
stripAnsiCodes([as], field, [type])data-manipulation, stringfield Removes ANSI color codes and movement commands.
subnet([as], bits, field)networkfield Computes a subnet from a IPV4 field.
sum([as], field)aggregatefield Calculates the sum for a field over a set of events.
table(fields, [limit], [order], [reverse], [sortby], [type])widgetfields Used to create a widget to present the data in a table.
tail([limit])eventlimit Returns the newest events.
test(expression)filter, eventexpression Evaluates boolean expression and filters events.
time:dayOfMonth([as], [field], [timezone], [timezoneField])time-datefield Gets the day of the month of a timestamp field.
time:dayOfWeek([as], field, [timezone], [timezoneField])time-datefield Gets day of week from 1 (Mon) to 7 (Sun) of a timestamp.
time:dayOfWeekName([as], [field], [timezone], [timezoneField])time-datefield Gets the English display name of day of the week of a timestamp field.
time:dayOfYear([as], [field], [timezone], [timezoneField])time-datefield Gets the day of the year of a timestamp field, from 1 to 365, or 366 in a leap year.
time:hour([as], [field], [timezone], [timezoneField])time-datefield Gets the hour (24-hour clock) of a timestamp field.
time:millisecond([as], [field], [timezone], [timezoneField])time-datefield Gets the millisecond of a timestamp field.
time:minute([as], [field], [timezone], [timezoneField])time-datefield Gets the minute value of a timestamp field.
time:month([as], [field], [timezone], [timezoneField])time-datefield Gets the month of a timestamp field (from 1 to 12).
time:monthName([as], [field], [timezone], [timezoneField])time-datefield Gets the English name of month of a timestamp field (e.g., January).
time:second([as], [field], [timezone], [timezoneField])time-datefield Gets the second of a timestamp field.
time:weekOfYear([as], [field], [timezone], [timezoneField])time-datefield Gets the week number within a year of a timestamp, a value from 1 to 53.
time:year([as], [field], [timezone], [timezoneField])time-datefield Gets the year of a timestamp field.
timeChart([buckets], [function], [limit], [minSpan], [series], [span], [timezone], [unit])widgetseries Used to draw a linechart where the x-axis is time.
tokenHash([as], field)stringfield Calculates a structure hash which is equal for similarly structured input.
top([as], [error], field, [limit], [max], [percent], [rest], [sum])aggregatefield Finds the top results based on a given field.
transpose([column], [header], [limit], [pivot])data-manipulation, stringpivot Transposes a query results set by creating an event for each attribute.
unit:convert([as], [binary], field, [from], [keepUnit], [to], [unit])data-manipulation, mathfield Converts values between different units.
upper([as], field, [locale])formatting, data-manipulationfield Changes contents of a string field to upper-case letters.
urlDecode([as], field)networkfield URL-decodes the contents of a string field.
urlEncode([as], field, [type])data-manipulation, networkfield URL-encodes the contents of a string field.
window([buckets], [function], [span])aggregatefunction Computes aggregate functions over a sliding window of data.
worldMap([ip], [lat], [lon], [magnitude], [precision])widgetworldmap Used to produce data compatible with the World Map widget.
writeJson([as], [field])data-manipulation, formatfield Writes data, including fields, as a JSON object.
xml:prettyPrint([as], field, [step], [strict], [width])data-manipulationfield Nicer output to an XML field.