Makes an extra copy of the event, thus the next step in the
pipeline will see both events. This is mostly useful in the
parser pipeline.
For On-Prem deployments only: If you are using this function to
copy an event to another repository, the
ALLOW_CHANGE_REPO_ON_EVENTS environment variable
must be set to true.
In this example, an event is stored with both the timestamp from the
event and a separate stream based on arrival time (assuming the event
has a type that is not arrivaltime).
Step-by-Step
Starting with the source repository events.
logscale
copyEvent("arrivaltime")
Creates a copy of the current event, and assigns the type
arrivaltime to the copied event.
logscale
|case{#type=arrivaltime
Returns a specific value that meets the defined condition. In this case,
it checks if the event type is
arrivaltime, then categorizes
all events by their arrivaltimes.
logscale
|@timestamp:=now();*
Sets the @timestamp field to the current time
now() for all events of the type
arrivaltime, and adds the
; separator and * to ensure, that
all other fields are kept unchanged. As the now()
is placed after the first aggregate function, it is evaluated
continuously, and returns the live value of the current system time,
which can divert between LogScale nodes.
logscale
|parseTimestamp(field=ts)}
As the original events keep the original timestamp, it parses the
timestamp from a field named ts
for events that are not of type
arrivaltime.
Event Result set.
Summary and Results
The query is used to make an extra copy of an event, when parsed, both
copies will be visible in the pipeline. The query creates a copy with
type arrivaltime, and sets its
timestamp to the current time, while the original event retains its
original timestamp. This allows tracking both when an event occurred
(original timestamp) and when it was received/processed (arrival time).
The query is useful in log processing and data management.
Make Copy of Events from One Repo to Another Repo
Use one parser to ingest data into multiple repositories
In this example, an event is copied from one repo to another and the
copied event can only be used in a parser
.
Step-by-Step
Starting with the source repository events.
logscale
copyEvent("cloned_event")
Creates a copy of the current event, and assigns the type
cloned_event to the copied
event. Now two events are flowing through the parser, one event
containing the field
cloned_event, and one event
without that field. In other words, it creates a copy with the type
cloned_event and assigns it to a
different repository.
logscale
|case{#type="cloned_event"
Returns a specific value that meets the defined condition. In this case,
it checks if the event type is
cloned_event. The case construct
is used to direct the two events to a different target repo.
logscale
|repo :="target-repo-name";*}
Creates a new repo named
target-repo-name with all events
of the type cloned_event being
directed. The use of * ensures, that all other fields
are kept unchanged.
Event Result set.
Summary and Results
The query is used to ingest data into multiple repositories using the
same parser. Shipping all data to one parser and having that parser ship
data to many different repositories can be useful: for example, if logs
are being sent from a single source, it is possible to setup one parser
that can parse all events from this source and decide which repositories
to send events to.
For On-Prem deployments only: If you are using this function to copy
an event to another repository, the
ALLOW_CHANGE_REPO_ON_EVENTS environment variable must
be set to true.
