S3 Ingest Self-hosted Preparation

Note

Changes to your LogScale Self-Hosted Configuration should be planned and reviewed with a LogScale Professional Services consultant. Please not that the Support team does not help plan and deploy configuration changes, product Support is intended for functionality that is already deployed.

Configuring S3 ingest feed for self install requires some additional steps to grant the required access and enable ingest feeds.

  1. Create an IAM Role in AWS. This role will need permissions allowing sts:AssumeRole on the roles that will be used for AWS S3 ingest.

    The following are examples of the trust and permission policies which must be applied to the role in AWS.

    Trust policy for the IAM role 

    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "TrustLogScale",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:user/example-logscale-user"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

    Permission policy for the IAM role 

    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowAssumingAllRoles",
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Resource": "arn:aws:iam::*:role/*"
        }
    ]
}

  2. Configure the INGEST_FEED_AWS_ROLE_ARN environment variable with the ARN of the Role that LogScale should assume.

  3. LogScale must be configured with AWS credentials. LogScale will resolve AWS credentials by (in order of precedence):

    • Reading the INGEST_FEED_AWS_ACCESS_KEY_ID and INGEST_FEED_SECRET_ACCESS_KEY environment variables.

      If used, these variables would have have been generated in AWS either by hand or using a provisioning tool and added to the list of environment variables as part of the Humio deployment.

    • Using the provider chain applied by LogScale is documented here:Default Credentials Provider Chain from the AWS Java SDK, note that we do not apply step 2 (Environment Variables).

Important

Changes made to the configuration files must be made on all nodes, and each node will need to be restarted for the changes to take effect.