Export to S3 Bucket with Amazon

By default LogScale allows downloading the results of a query to a file. This file is generated as a HTTP stream directly from LogScale, and can be long-lasting with long periods of no data being transmitted when LogScale is searching for rare hits in large data sets. This can cause issues for some networks and load balancers.

As an alternative, LogScale allows exporting to Amazon S3. The result of the query will be uploaded to the bucket storage provider and the user will be given a URL to download the file once the upload is complete.

As LogScale uses signed URLs for downloads the user does not need read access to the bucket, LogScale needs write access to the target bucket.

Following is the most basic example configuration for exporting to Amazon S3:

ini
# The credentials used to authenticate with
S3_EXPORT_ACCESSKEY=$ACCESS_KEY
S3_EXPORT_SECRETKEY=$SECRET_KEY

# The name of the region and bucket to export to
S3_EXPORT_REGION=$BUCKET_REGION
S3_EXPORT_BUCKET=$BUCKET_NAME

LogScale supports many other kinds of authentication on AWS, as described earlier on this page.

HTTP Proxy

If LogScale is set up to use an HTTP proxy, it will per default be used for communicating with S3. It can be disabled using the following:

ini
# Use the globally configured HTTP proxy for communicating with S3.
# Default is true.
S3_EXPORT_USE_HTTP_PROXY=false