Ingesting Data from AWS S3
Security Requirements and Controls
Change ingest feeds
permission
CrowdStrike Falcon LogScale has the ability to ingest logs from AWS S3 buckets, which can then be managed in LogScale and leveraged using queries, alerts and alarms. In the following we will run through the configuration process of ingesting this data.
Amazon Web Services log data is an extremely valuable data source that comes in a variety of flavors depending on the services you are looking to learn more about. Some of the most common data sources include AWS VPC flow™, Cloudtrail™ and CloudWatch™. These logs can be directed to S3 buckets where they are often ingested by LogScale.
For more details on these logs, see here.
Prerequisites for Ingesting AWS Data
To follow these steps, you will need:
Access to AWS and basic knowledge of AWS architecture. You will also need to configure your source in AWS to log to an S3 bucket refer to documentation. This can be to a separate bucket or a directory within a bucket. These log files will then be pulled into Falcon LogScale for analysis and visualization, the format of the data can be line-delimited or AWS JSON events. AWS example events are referenced here.
Access to a Falcon LogScale environment, with a repository where you want to ingest the data.
Change ingest feed
permissionPerformed the S3 Ingest Self-hosted Preparation before using ingest feeds in self hosted scenarios.
Once this these requirements have been satisfied, you are ready to follow Setting up a New Ingest Feed and configure your ingest feeds.
Monitoring Ingest Feeds
Errors during ingest will be shown within the Ingest
feeds
page.
The page shows the individual feeds, their configuration and current status:
Name
Name of the ingest feed given during configuration.
Preprocessing
A description of any preprocessing performed on the data before it is provided to the parser.
Parser
The configured parser for this ingest feed.
Last activity
The time of the last ingest activity for this feed.
Status
The current status of the feed, including whether there is an error or if the feed has been disabled.
Transient errors during ingest should automatically be cleared. You can click on the status message to get more information.
To edit an existing feed, use the ⋮ next to each feed configuration. To create a new feed, blick the button.