Query Functions

Manipulating and formatting functions for extracting information from your event data

LogScale query functions take a set of events, parameters, or configurations. They produce, reduce, or modify values within that set, or in the events themselves within a query pipeline.

Below is an alphabetical listing of all LogScale query functions:

Table: Query Functions

FunctionTypeDefault ArgumentAvailabilityDescription
array:contains(array, value)array, data-manipulation, filterarray  Checks whether the given value matches any of the values of the array and excludes the event if no value matches
array:filter(array, [asArray], function, var)array, data-manipulationarray  Drops entries from the input array using the given filtering function.
array:reduceAll(array, function, var)aggregate, array, data-manipulationarray  Computes a value from all events and array elements of the specified array.
array:regex(array, [flags], regex)array, filter, regular-expressionarray  Checks whether the given pattern matches any of the values of the array and excludes the event from the search result
asn([as], [field])networkfield  Determines autonomous system number and organization associated.
avg([as], field)aggregate, statisticsfield  Calculates the average for a field of a set of events.
base64Decode([as], [charset], field)hash, parsingfield  Performs Base64 decoding of a field.
bitfield:extractFlags(field, [onlyTrue], output)data-manipulation   Interprets an integer as a bit field and extracts the specified flags
bucket([buckets], [field], [function], [limit], [minSpan], [span], [timezone], [unit])aggregatespan  Extends the groupBy() function for grouping by time.
callFunction([as], field, function)aggregatefunction  Calls the named function on a field over a set of events.
cidr([column], field, [file], [negate], [subnet])filter, networkfield  Filters events using CIDR subnets.
collect(fields, [limit], [multival], [separator])aggregatefields  Collects fields from multiple events into one event.
communityId([as], destinationip, [destinationport], [icmpcode], [icmptype], proto, [seed], sourceip, [sourceport])network   Computes the Community ID, a standard for hashing network flows.
concat([as], field)data-manipulation, stringfield  Concatenates the values of a list of fields into a value in a new field.
concatArray([as], field, [from], [prefix], [separator], [suffix], [to])array, data-manipulation, stringfield  Concatenates values of all fields with same name and an array suffix into a new field.
copyEvent(type)eventtype  Duplicates event so pipeline will see both events.
count([as], [distinct], [field])aggregate, statisticsfield  Counts given events.
counterAsRate([as], field)aggregatefield  Calculates the rate for a counter field.
createEvents(rawstring)aggregate-testing, eventrawstring  Generates temporary events as part of the query.
crypto:md5([as], field)hashfield  Computes a cryptographic MD5-hashing of an input string
default(field, [replaceEmpty], value)event, formatvalue  Creates a field to given parameter and given value.
drop(fields)data-manipulation, eventfields  Removes attributes or columns from a result set.
dropEvent()event   Drops completely an event in parser pipeline to stop it from being ingested.
end([as])time-dateas  Assign the end of the search time interval to the field provided by parameter as.
eval()data-manipulation   Creates a new field by evaluating the provided expression.
eventFieldCount([as])event   Computes number of fields event uses internally for the values.
eventInternals([prefix])event   Add a set of fields describing the storage locations of this event.
eventSize([as])event   Determines the number of bytes that this event uses internally for the values, not counting the bytes for storing the field names.
fieldset()event   Retrieves a list of available fields.
fieldstats([limit])aggregate, event   Retrieves stats about fields.
findTimestamp([addErrors], [as], [field], [timezone], [timezoneAs])time-date   Finds timestamp in given field and parses, trying multiple timestamp formats.
format([as], field, format, [timezone])data-manipulation, formatformat  Formats a string using printf-style.
formatDuration([as], field, [from], [precision])format, time-datefield  Formats a duration into a more readable string.
formatTime(as, [field], format, [locale], [timezone])formatformat  Formats a string according to strftime().
geohash([as], [lat], [lon], [precision])geolocation   Calculates a geohash value given two fields representing latitude and longitude.
groupBy(field, [function], [limit])aggregatefield  Groups events by specified fields and executes aggregate functions on each group.
hash([as], field, [limit], [seed])event, hashfield  Computes a non-cryptographic hash of a list of fields.
hashMatch([bits], [field], [hash], input, [salt])hash, securityinput  Calculates a secure hash of a field and uses it to match events as a filter.
hashRewrite([as], [bits], field, [hash], [replaceInRawstring], salt)hash, securityfield  Calculates a secure hash of a field for storing in the event.
head([limit])aggregatelimit  Finds the oldest events.
in(field, ignoreCase, values)comparison, filterfield  Filters records by values where field is in given values.
ioc:lookup([confidenceThreshold], field, [include], [prefix], [strict], type)securityfield  Look up IOCs (Indicators of Compromise).
ipLocation([as], [field])geolocationfield  Determines country, city, longitude, and latitude for given IP address.
join([end], field, [include], [key], [limit], [live], [max], [mode], query, [repo], [start], [view])joinquery  Join two LogScale searches.
json:prettyPrint([as], [field], [step], [strict])data-manipulation, formatfield  Nicer output to a JSON field.
kvParse([as], [excludeEmpty], [field], [override], [separator], [separatorPadding])parsingfield  Key-value parse events.
length([as], field)stringfield  Computes the number of characters in a string field.
linReg([prefix], x, y)aggregate   Computes linear relationship model between two variables using least-squares fitting.
lower([as], field, [locale], [type])formatfield  Changes text of a given string field to lower-case letters.
lowercase(field, [include], [locale])data-manipulation, format, stringfield  Changes field name or content to lowercase for parsers.
match([column], field, file, [glob], [ignoreCase], [include], [mode], [strict])filter, stringfile  Searches text using a CSV or JSON file and can enhance entries.
math:abs([as], field)mathfield  Calculates the absolute value of a field; the result is always a positive number or 0.
math:arccos([as], field)mathfield  Calculates the arc cosine of a field.
math:arcsin([as], field)mathfield  Calculates the arc sine of a field.
math:arctan([as], field)mathfield  Calculates the arc tangent of a value.
math:arctan2([as], x, y)math   Calculates the arc tangent of a value.
math:ceil([as], field)mathfield  Rounds field value to smallest integer that's larger than or equal to it.
math:cos([as], field)mathfield  Calculates the cosine of a field.
math:cosh([as], field)math   Computes the hyperbolic cosine of a double field.
math:deg2rad([as], field)mathfield  Converts angles from degrees to radians.
math:exp([as], field)mathfield  Calculates Euler's number e raised to the power of a double value in a field.
math:expm1([as], field)mathfield  Calculates the exponential value of a number minus 1.
math:floor([as], field)mathfield  Computes the largest integer value not greater than the field value given.
math:log([as], field)mathfield  Calculates the natural logarithm (base e) of the value in a double field.
math:log10([as], field)mathfield  Calculates the base 10 logarithm of a double field.
math:log1p([as], field)mathfield  Calculates the natural logarithm of the sum of field's value and 1.
math:log2([as], field)mathfield  Calculates the base 2 logarithm of a double field.
math:mod([as], divisor, field)mathfield  Calculates the floor modulus of field value and the divisor.
math:pow([as], exponent, field)mathfield  Calculates the field value to the exponent power.
math:rad2deg([as], field)mathfield  Converts angles from radians to degrees.
math:sin([as], field)mathfield  Calculates the sine of a field.
math:sinh([as], field)mathfield  Calculates the hyperbolic sine of a double field.
math:sqrt([as], field)mathfield  Calculates the rounded positive square root of a double field.
math:tan([as], field)mathfield  Calculates the trigonometric tangent of an angle in a field.
math:tanh([as], field)mathfield  Calculates the hyperbolic tangent of a field.
max([as], field, [type])statisticsfield  Finds the largest number for the specified field over a set of events.
min([as], field, [type])statisticsfield  Finds the smallest number for the specified field over a set of events.
now([as])time-dateas  Assign the current time to the field provided by parameter as.
parseCEF([field], [headerprefix], [keeplabels], [labelprefix], [prefix])data-manipulation, parsingfield  Parses CEF version 0.x encoded messages.
parseCsv(columns, [delimiter], [excludeEmpty], field)data-manipulation, parsingfield  Parses a CSV-encoded field into known columns.
parseFixedWidth(columns, [field], [trim], widths)parsingfield  Parses a fixed width-encoded field into known columns.
parseHexString([as], [charset], field)data-manipulation, math, parsingfield  Parses input from hex encoded bytes, decoding resulting bytes as a string.
parseInt([as], [endian], field, [radix])math, parsingfield  Converts an integer from any radix or base to base-ten, decimal radix.
parseJson([exclude], [excludeEmpty], field, [handleNull], [include], [prefix], [removePrefixes])data-manipulation, parsingfield  Parses specified fields as JSON.
parseLEEF([delimiter], [field], [headerprefix], [keeplabels], [labelprefix], [parsetime], [prefix], [timezone])data-manipulation, parsingfield  Parses LEEF version 1.0 and 2.0 encoded messages.
parseTimestamp([addErrors], [as], [caseSensitive], field, [format], [timezone], [timezoneAs])parsing, time-dateformat  Parses a string into a timestamp.
parseUrl([as], [field])parsingfield  Extracts URL components from a field.
parseXml(field, [prefix], [strict])parsingfield  Parses specified field as XML.
percentile([accuracy], [as], field, [percentiles])aggregate, statisticsfield  Finds one event with a field for each percentile specified.
range([as], field)statisticsfield  Finds numeric range between smallest and largest numbers for field over a set of events.
rdns([as], field, [server])aggregate, networkfield  Events using RDNS lookup.
regex([field], [flags], [limit], regex, [repeat], [strict])filter, regular-expression, stringregex  Extracts new fields using a regular expression.
rename([as], field)data-manipulation, eventfield  Renames one or more given fields.
replace([as], [field], [flags], regex, [replacement], [with])data-manipulation, regular-expression, stringregex  Replaces each substring that matches given regular expression with given replacement.
round([as], field, [how])mathfield  Rounds an input field up or down, depending on which is nearest.
sample([field], [percentage])filter, statisticspercentage  Samples the event stream.
sankey(source, target, [weight])aggregate, data-manipulation, widget   Produces data compatible with Sankey widget.
select(fields)eventfields  Used to specify a set of fields to select from each event.
selectFromMax(field, include)aggregatefield  Selects event with the largest value for the specified field.
selectFromMin(field, include)aggregatefield  Selects event with the smallest value for the specified field.
selectLast(fields)aggregatefields  Specify fields to select from events, keeping value of most recent event for each field.
selfJoin([collect], field, [limit], [postfilter], [prefilter], [select], where)joinfield  Used to collate data from events that share a key.
selfJoinFilter(field, [prefilter], where)filter, joinfield  Runs query to determine IDs, and then gets all events containing one of them.
series(collect, [endmatch], [maxduration], [maxpause], [memlimit], [separator], [startmatch])aggregate, data-manipulationcollect  Collects a series of values for selected fields from multiple events into one or more events.
session([function], [maxpause])aggregate, statisticsfunction  Collects events into sessions, and aggregates them.
shannonEntropy([as], field)networkfield  Calculates a entropy measure from a string of characters.
sort([field], [limit], [order], [reverse], [type])aggregatefield  Sorts events by their fields.
split([field], [strip])array, data-manipulation, regular-expression, stringfield  Splits an event structure created by a JSON array into distinct events.
splitString([as], by, [field], [index])array, data-manipulation, stringfield  Splits a string by specifying a regular expression by which to split.
start([as])time-dateas  Assign the start of the search time interval to the field provided by parameter as.
stats([function])aggregate, statisticsfunction  Used to compute multiple aggregate functions over the input.
stdDev([as], field)statisticsfield  Calculates the standard deviation for a field over a set of events.
stripAnsiCodes([as], field)data-manipulation, stringfield  Removes ANSI color codes and movement commands.
subnet([as], bits, field)networkfield  Computes a subnet from a IPV4 field.
sum([as], field)aggregatefield  Calculates the sum for a field over a set of events.
table(fields, [limit], [order], [reverse], [sortby], [type])aggregate, widgetfields  Used to create a widget to present the data in a table.
tail([limit])aggregatelimit  Finds the newest events.
test(expression)comparison, event, filterexpression  Evaluates boolean expression and filters events.
time:dayOfMonth([as], [field], [timezone], [timezoneField])time-datefield  Gets the day of the month of a timestamp field.
time:dayOfWeek([as], field, [timezone], [timezoneField])time-datefield  Gets day of week from 1 (Mon) to 7 (Sun) of a timestamp.
time:dayOfWeekName([as], [field], [timezone], [timezoneField])time-datefield  Gets the English display name of day of the week of a timestamp field.
time:dayOfYear([as], [field], [timezone], [timezoneField])time-datefield  Gets the day of the year of a timestamp field, from 1 to 365, or 366 in a leap year.
time:hour([as], [field], [timezone], [timezoneField])time-datefield  Gets the hour (24-hour clock) of a timestamp field.
time:millisecond([as], [field], [timezone], [timezoneField])time-datefield  Gets the millisecond of a timestamp field.
time:minute([as], [field], [timezone], [timezoneField])time-datefield  Gets the minute value of a timestamp field.
time:month([as], [field], [timezone], [timezoneField])time-datefield  Gets the month of a timestamp field (from 1 to 12).
time:monthName([as], [field], [timezone], [timezoneField])time-datefield  Gets the English name of month of a timestamp field (e.g., January).
time:second([as], [field], [timezone], [timezoneField])time-datefield  Gets the second of a timestamp field.
time:weekOfYear([as], [field], [timezone], [timezoneField])time-datefield  Gets the week number within a year of a timestamp, a value from 1 to 53.
time:year([as], [field], [timezone], [timezoneField])time-datefield  Gets the year of a timestamp field.
timeChart([buckets], [function], [limit], [minSpan], [series], [span], [timezone], [unit])aggregate, widgetseries  Used to draw a linechart where the x-axis is time.
tokenHash([as], field)hash, stringfield  Calculates a structure hash which is equal for similarly structured input.
top([as], [error], field, [limit], [max], [percent], [rest], [sum])aggregatefield  Finds the top results based on a given field.
transpose([column], [header], [limit], [pivot])aggregate, data-manipulation, stringpivot  Transposes a query results set by creating an event for each attribute.
unit:convert([as], [binary], field, [from], [keepUnit], [to], [unit])data-manipulation, mathfield  Converts values between different units.
upper([as], field, [locale])data-manipulation, formatfield  Changes contents of a string field to upper-case letters.
urlDecode([as], field)networkfield  URL-decodes the contents of a string field.
urlEncode([as], field, [type])data-manipulation, networkfield  URL-encodes the contents of a string field.
wildcard([field], [ignoreCase], pattern)filterpattern  Performs a wildcard pattern search with optional case insensitivity.
window([buckets], [function], [span])aggregatefunction  Computes aggregate functions over a sliding window of data.
worldMap([ip], [lat], [lon], [magnitude], [precision])aggregate, widget   Used to produce data compatible with the World Map widget.
writeJson([as], [field])data-manipulation, formatfield  Writes data, including fields, as a JSON object.
xml:prettyPrint([as], field, [step], [strict], [width])data-manipulationfield  Nicer output to an XML field.