Make Copy of Events

Make an extra copy of the event to be parsed along with the original event using the copyEvent() function

Query

logscale

copyEvent("arrivaltime")
| case { #type=arrivaltime
| @timestamp:=now() ; *
| parseTimestamp(field=ts) }

Introduction

The copyEvent() function is used to make an extra copy of an event, when parsed, both copies will be visible in the pipeline. A common use of case statements is to return a specific value depending on a column's value in the result set.

In this example, an event is stored with both the timestamp from the event and a separate stream based on arrival time (assuming the event has a type that is not arrivaltime).

Step-by-Step

Starting with the source repository events.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[\Add Field/] 1[/Filter/] 2{{Aggregate}} 3[\Add Field/] result{{Result Set}} repo --> 0 0 --> 1 1 --> 2 2 --> 3 3 --> result style 0 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
copyEvent("arrivaltime")
```
Creates a copy of the current event, and assigns the type arrivaltime to the copied event.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[\Add Field/] 1[/Filter/] 2{{Aggregate}} 3[\Add Field/] result{{Result Set}} repo --> 0 0 --> 1 1 --> 2 2 --> 3 3 --> result style 1 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
| case { #type=arrivaltime
```
Returns a specific value that meets the defined condition. In this case, it checks if the event type is arrivaltime, then categorizes all events by their arrivaltimes.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[\Add Field/] 1[/Filter/] 2{{Aggregate}} 3[\Add Field/] result{{Result Set}} repo --> 0 0 --> 1 1 --> 2 2 --> 3 3 --> result style 2 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
| @timestamp:=now() ; *
```
Sets the @timestamp field to the current time now() for all events of the type arrivaltime, and adds the ; separator and * to ensure, that all other fields are kept unchanged. As the now() is placed after the first aggregate function, it is evaluated continuously, and returns the live value of the current system time, which can divert between LogScale nodes.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 0[\Add Field/] 1[/Filter/] 2{{Aggregate}} 3[\Add Field/] result{{Result Set}} repo --> 0 0 --> 1 1 --> 2 2 --> 3 3 --> result style 3 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
| parseTimestamp(field=ts) }
```
As the original events keep the original timestamp, it parses the timestamp from a field named ts for events that are not of type arrivaltime.
Event Result set.

Summary and Results

The query is used to make an extra copy of an event, when parsed, both copies will be visible in the pipeline. The query creates a copy with type arrivaltime, and sets its timestamp to the current time, while the original event retains its original timestamp. This allows tracking both when an event occurred (original timestamp) and when it was received/processed (arrival time). The query is useful in log processing and data management.

Examples Library