Amazon GuardDuty (Preview)

Amazon GuardDuty is a security monitoring service that analyzes and processes foundational data sources, such as AWS CloudTrail management events, AWS CloudTrail event logs, VPC flow logs (from Amazon EC2 instances), and DNS logs.

It also processes Kubernetes audit logs, RDS login activity, S3 logs, EBS volumes, Runtime monitoring, and Lambda network activity logs. Ingest GuardDuty logs directly from your S3 bucket into a LogScale repository to get faster and more comprehensive early detections for potential attach indicators.

The parser normalizes data to a common schema based on an OpenTelemetry standard. This schema allows you to search the data without knowing the data specifically, and just knowing the common schema instead. It also allows you to combine the data more easily with other data sources which conform to the same schema.

Preview Status

Note that this package is considered a PREVIEW. This means we are seeking feedback on the package, and may make breaking changes to the parser in the future. It means also it may not include all documentation which usually we publish for packages.

Given the preview status of this package there are some limitations to what the parser is able to do. Please check out the package on Marketplace to see the most recent list of limitations.

Installing the Package in LogScale

Find the repository where you want to send the Amazon GuardDuty (Preview) events, or Creating a Repository or View.

  1. Navigate to your repository in the LogScale interface, click Settings and then Packages on the left.

  2. Click Marketplace and install the LogScale package for Amazon GuardDuty (Preview) (i.e. aws/guardduty).

Configurations and Sending the Logs to LogScale

  1. To get logs from AWS GuardDuty into LogScale, configure the export of GuardDuty findings to an S3 bucket.

  2. Then you need to configure LogScale to collect data from AWS S3 buckets using the guardduty-json. See the documentation for cloud: Ingesting Data from AWS S3 and self-hosted Ingesting Data from AWS S3 deployments to send logs directly from S3 bucket into LogScale repository.

Verify Data is Arriving in LogScale

Once you have completed the above steps the Amazon GuardDuty (Preview) data should be arriving in your LogScale repository.

You can verify this by doing a simple search for the events:

#Vendor = "amazon" 
| Product="guardduty"

Package Contents Explained

This package parses incoming data, and normalizing the data as part of that parsing. The parser normalizes the data to a subset of this schema based on OpenTelemetry standards, while still preserving the original data.

If you want to search using the original field names and values, you can access those in the fields whose names are prefixed with the word Vendor. Fields which are not prefixed with Vendor are standard fields which are either based on the schema (e.g. source.ip) or on LogScale conventions (e.g. @rawstring).

The fields which the parser currently maps the data to, are chosen based on what seems the most relevant, and will potentially be expanded in the future. But the parser won't necessarily normalize every field that has potential to be normalized.

Event Categorisation

As part of the schema, events are categorized by fields:

  • event.kind

  • event.type

event.type is an array so needs to be searched using the following syntax:

array:contains("event.type[]", value="info")

For example, the following will find events where some event.type[n] field contains the value network, regardless of what n is.

Note that not all events will be categorized to this level of detail.

Normalized Fields

Here are some of the normalized fields which are being set by this parser:

  • event.* (e.g. event.type, event.kind,, event.start, event.end, event.severity )

  • cloud.* (e.g., cloud.provider, cloud.region, cloud.service )

  • source.* (e.g. source.ip, source.port, source.address )

  • destination.* (e.g. destination.ip, destination.address )

  • user.* (e.g. user.ip, )

  • related.* (e.g. realted.ip, related.user )