Sinks

Sinks are the destination of the data being collected, the Falcon LogScale Collector is designed to send data to LogScale only. It makes use of LogScale's proprietary ingest APIs as these have been optimized for efficient transport of event data including features like hierarchical metadata.

You can define multiple sinks for each configuration file. See Sinks (sinks) for more information.

The LogScale ingest APIs currently transport data over HTTP to the same ports that are used for the web interface for LogScale, no special ports need to be configured. By default the data is compressed and requires HTTPS, although these can be configured.

The LogScale Collector also supports custom TLS configuration, and HTTP(S) proxies as required.

Buffering

The Falcon LogScale Collector buffers events before sending them to LogScale. This allows the LogScale Collector to optimize between efficient batch sizes and minimal ingest lag. For input types where the data cannot be re-read (syslog, and exec) these buffers also provide some durability for the data.

Metadata

To ensure the data that comes from the LogScale Collector is useful we attach metadata to all the events that are sent. The exact metadata that is sent depends on the source, but everything is prefixed with @collect.*, this includes details about the host that sent the event, etc.