Data Sources

Data sources are the data points from which the data is collected. The Falcon Logscale Collector currently supports the following inputs or data sources:

Collecting Events from Files

Collecting events from local files on disk is one of the most common log collection scenarios. Examples include logs produced by custom applications, web servers, and firewalls.

  • Glob pattern to specify the file(s) to collect; recursively collect files from a directory

  • Glob pattern to exclude files

  • Sends the entire existing content of files it finds

  • Tails existing files looking for new events

  • Multiline logs

  • Handles log rotation scenarios

Collecting Windows Events

Collecting Windows Events is simple and produces rich events. The Log Collector attempts to automatically detect which channels are available, or you can explicitly identify which channels you want to collect.

The Log Collector uses the internal Windows events templates to ensure the event is fully parsed where possible; this means that not only can you see the human readable representation of the event, you get all fields parsed automatically and the XML representation of the event.

Syslog Receiver

Collecting TCP and UDP syslog streams from within the infrastructure is an important feature in securing legacy logging scenarios. The Log Collector can listen for unencrypted TCP or UDP syslog traffic on any port and will receive and buffer that data and stream it securely to LogScale.

Native TLS-encrypted syslog ingest is not supported in the Log Collector at this time. Deploying the Log Collector as close to the system sending syslog minimizes exposure to the unsecured traffic, and also provides maximum durability for syslog over UDP.

The Log Collector does not have side effects in this scenario, and doesn't tamper with the events in any way (i.e. no manipulation of the syslog headers), but does provide additional useful metadata on the events outside of the syslog envelope.

Exec Input

The Log Collector supports running a user configured subprocess to gather log data. This process is run based on a schedule and all the output produced by the subprocess on stderr and stdout is streamed to LogScale as events.

This allows the Log Collector to gather any information from the host that is available from the standard tools, or administrators can provide a script.

This custom input type can be used to extend the Log Collector to check host metrics, perform ping and HTTP based polling, or pull data from any other kind of API or service.

Collecting Logs from SystemD on Linux

The journald source collects systemd logs from a local linux journal. The structured journal has some advantages compared to plain text files, including built in filtering on specific systemd units, reading logs from the current boot only and built in log rotation.

The output of the source is similar, depending on the configuration, to what you would see with the journal viewer journalctl.