Popular Ingest Methods
Falcon LogScale Collector provides you with visibility and insights across your organization's computing environment by centralizing your logs and events in one place. Log Collector is able to provide these insights through flexible data ingest options, which provide a means of collecting logs from a wide range of data sources, such as Kafka, Syslog, or Windows event logs.
The ability to ingest logs and events from many sources provides comprehensive visibility across your entire infrastructure which, in turn, enables faster incident response and troubleshooting.
LogScale's ingest capabilities are designed to handle extremely large data volumes with minimal overhead, allowing you to collect everything without sampling, and while maintaining real-time search performance.
To help you get started, the following table summarizes the popular ingest methods covered in this section. Use this information to help you decide which types of events and logs you should collect, and which ingest methods will work best for you. Click the links in the table to learn more about each ingest method:
| Method | Description | Key Points |
|---|---|---|
| Falcon LogScale Collector |
Lightweight, purpose-built agent designed to collect log data from various sources on endpoints, servers, and containers, and forward it efficiently to Falcon LogScale Collector. |
|
| Amazon S3 Bucket |
Ingest logs from AWS S3 buckets through S3 event notifications, enabling centralized analysis of CloudTrail, VPC Flow, and other AWS service logs. |
|
| Azure Event Hubs |
Fully managed, real-time data ingestion service capable of receiving and processing millions of events per second with low latency. |
|
| HTTP / HTTPS API |
Direct, flexible method for sending log data and events to Falcon LogScale Collector using standard HTTP POST requests from applications, scripts, and services. |
|
| Syslog |
Universal protocol supported by a wide range of devices and operating systems for sending system log or event messages to a central server. |
|
| Kafka |
Distributed event streaming platform designed for high-throughput, fault-tolerant handling of real-time data feeds. |
|
| Filebeat |
Lightweight shipper from Elastic that forwards and centralizes log data from servers, containers, and applications. |
|
| Logstash |
Free and open-source data processing pipeline from Elastic that ingests data from multiple sources simultaneously, transforms it, and sends it to various destinations. |
|
| Fluentd |
Open-source data collector that unifies log collection and consumption, enabling you to aggregate logs from multiple sources, transform them, and route them to various destinations. |
|
| Google Cloud Logging |
Fully managed service for storing, searching, analyzing, monitoring, and alerting on log data and events from Google Cloud Platform (GCP) and AWS. |
|
| Windows Event Collector |
Windows service that collects events from Windows Event Logs on remote computers and forwards them to a central collector. |
|
| Database Logs (JDBC) |
JDBC-based log ingestion allows Falcon LogScale Collector to directly query database tables and views to collect log data and audit trails. |
|
| SNMP Traps |
Asynchronous notifications sent by network devices, servers, or applications to report significant events or state changes. |
|
Each method is described in detail in the sections that follow, including setup instructions, configuration examples, and best practices.