Function Input/Output
Before looking at the specifics of the different Function Types, functions operate as part of the pipeline of processing that occurs when a query is executed, as detailed in Query Pipeline. The operation of different functions falls into three broad categories:
Event Modifying Functions
Functions that modify each event in the event set; for example adding a field, or modifying/updating the value of a field in each event.
For example, given the input data:
user kingm edwardst howardn woodm walkerd blackj rogersd thomasb stevensp The query:
logscaleupper(user,as=upperuser)Creates a new field, upperuser:
user upperuser kingm KINGM edwardst EDWARDST howardn HOWARDN woodm WOODM walkerd WALKERD blackj BLACKJ rogersd ROGERSD thomasb THOMASB stevensp STEVENSP When modifying events in this way all of the event data and fields for each event is retained.
Event Creation Functions
Functions that add new events to the event set, or that remove events from the event set. This has the affect of changing the list of events in each set. For example, using
split()to split an array duplicates the original event for each element of the array. So with this input data (where each is row is an event):permlist[0] permlist[1] permlist[2] user pid cmd engineering reliability performance evansm 43219 /usr/bin/ssh sales renewals <no value> edwardst 25678 /sbin/iptables product innovation <no value> davisr 50123 /usr/bin/curl Shows three events, and using:
logscalesplit(permlist)permlist user pid cmd performance evansm 43219 /usr/bin/ssh reliability evansm 43219 /usr/bin/ssh engineering evansm 43219 /usr/bin/ssh renewals edwardst 25678 /sbin/iptables sales edwardst 25678 /sbin/iptables innovation davisr 50123 /usr/bin/curl product davisr 50123 /usr/bin/curl Now with a result set of seven events. The original list of events has been updated with the new set, similar to aggregation, but with similar information.
Other functions that perform this creation of events that may be similar to (or completely replace) the original set include
defineTable(), join functions, includingjoin(), andreadFile().Event Set Modifying Functions
Functions that replace the incoming event set with a new set of events and fields; most often this is due to an aggregation, but it might also be functions that create new events such as
copyEvent()or remove events such asdropEvent(), or joins where new events are composed based combination of two or more events.With
groupBy()for example, the input data:user pid cmd perms evansm 43219 /usr/bin/ssh engineering,reliability,performance edwardst 25678 /sbin/iptables sales,renewals davisr 50123 /usr/bin/curl product,innovation cooperp 11789 /bin/ps finance,billing clarkd 33456 /usr/bin/top operations,process_improvement brownr 57891 /sbin/ifconfig hr,diversity_inclusion blackj 20123 /usr/bin/find marketing,partner_marketing bakerm 45678 /bin/ls it,automation andersonk 27891 /usr/sbin/useradd engineering,ai,ml ... Â Â Â The events and fields are completely replaced with the aggregated dataset:
user _count adamsb 3 andersonk 3 bakerm 3 blackj 3 brownr 3 clarkd 3 cooperp 3 ... Â