The dropEvent() function can be used both
during queries and within the parser pipeline.
Depending on
usage, the function has different behavior:
If used during
parsing, the event is dropped and removed entirely from the
query output, meaning that the event data will not be stored in
Falcon LogScale.
If used within normal searching, the
dropEvent() function is simply an alias for
false - it behaves the same as false.
Note
The dropEvent() does not accept any
arguments. The dropEvent() has different
behaviour depending on usage in parser or in normal searches.
Click + next to an example below to get the full details.
Drop Events Based on Parsing JSON Value
Query
logscale
case{@rawstring="#*"|dropEvent();*}
Introduction
When parsing incoming data, it is sometimes the case that the data
includes 'commented' data, where,for example, the
# character is used to
identify comments in files rather than real data. This example
removes those lines from the ingest process during parsing using
the dropEvent() function to drop the entire
event from the ingest pipeline.
Step-by-Step
Starting with the source repository events.
logscale
case{@rawstring="#*"|dropEvent();
Starts a case statement, with the
first matching expression looking for the hash symbol in a line to
indicate that it could be removed, then dropping the entire event using
dropEvent()
logscale
*}
For all other lines, the case
expression matches all other events and lets them through.
Event Result set.
Summary and Results
This query is used to remove data at ingestion, in this example data
that matches a typical source construct (the comment). When used within
the parser pipeline, the dropEvent() function
ensures that the data is removed entirely from the query output, meaning
that the event data will not be stored in LogScale.
Ignore Commented Lines During Parsing by Dropping Events
The dropEvent() function is often used within
parsers to drop events during parsing that do not need to be
ingested. The following example shows how to filter events as part
of a parser by matching a particular field value from being
ingested.
Step-by-Step
Starting with the source repository events.
logscale
parseJson()
Parses the incoming data to identify JSON values and converts them into
a usable field.
logscale
|case{someField="some_value"|dropEvent();*}
Starts a case statement, with the
first matching expression identifying a field value in the extracted
JSON field from the returned results. Then drops the event. This has the
effect of terminating the parsing for this event, as there is no more
data to be processed.
logscale
|parseTimestamp(field=@timestamp)
Parses the timestamp from the @timestamp field for
all other events that do not match the JSON value.
Event Result set.
Summary and Results
This query is used to drop events at ingestion. When used within the
parser pipeline, the dropEvent() is a simple and
practical way of eliminating events during the parsing of incoming data.
