The files written on the backup drive are encrypted using a secret provided during LogScale configuration, and allow you to store the backup on a network drive where others may have read access.

LogScale is designed to support solutions like Amazon S3 for archiving. This is not a full backup, but will archive the data so that it can be re-ingested by another or the same LogScale server at some later date.

You can do a full backup using any backup software that is able to back up all the files in the LogScale data directory. Any chosen backup software needs to support "sparse files" to be efficient.

You can back up your LogScale installation by adding a special mounted directory when you run the Docker container. LogScale writes its backup files to this directory. Each LogScale node can have its own backup directory that is not shared with the other nodes. This is possible since it is not always feasible to create one backup directory that can hold all the data in a LogScale cluster. In the backup directory, each LogScale node will create a sub-directory with its node ID as name and use this directory.

First, create an empty directory on the host machine to store data for LogScale. You would do that like so:

$ mkdir /humio-backups-on-host

We recommend creating the backup directory on a different disk from the main LogScale data directory. Make the directory a mount point for a network drive or other similar separation from the main data drive.

Next, edit the LogScale configuration file to set the backup parameters. Add these lines to that file:

BACKUP_NAME=humio-backup
BACKUP_KEY=mysecretkey-myhost-+R+q(AB9QG86xZMCKGyj

LogScale encrypts all backups with a secret key that you provide. This means that you can safely store backups on an unencrypted disk, or send them over the Internet. Keep the secret key safe and store it in another place. You cannot recover the backup if you lose access to it. If you lose the secret, delete all the files in the backup, or provide a new location to backup to, and start over. LogScale will then write a fresh backup.

The next step is to run LogScale using the Docker run command with the following argument like so:

$ run -v /humio-backups-on-host:/backup

This maps the backups directory on the host. In this example, /humio-backups-on-host will send the backup to the /backup directory in the container. LogScale will then start backing up data to the specified directory.

The procedure is similar. Instead of mounting the directory using -v, you specify the location using BACKUP_DIR. A full example configuration is as follows:

BACKUP_NAME=humio-backup
BACKUP_KEY=mysecretkey-myhost-+R+q(AB9QG86xZMCKGyj
BACKUP_DIR=/mnt/my-net-server/humio-backup01

When is data deleted in backup? Right when retention kicks in and deletes data in LogScale. It is possible to configure a delay so that data is not deleted in the backup until some time has elapsed since it was deleted in LogScale. This is configured using DELETE_BACKUP_AFTER_MILLIS. By default LogScale will not delete data in the backup until seven days after the data was deleted in LogScale.

DELETE_BACKUP_AFTER_MILLIS=604800000

LogScale can restore all events that were stored in segment files for a LogScale node in the humio-data directory from the backup. It can also help a node claim the nodeID of a lost node in a cluster.

If your node is lost, but you have reinstalled the OS on the existing hardware, or perhaps found a spare server that will now take on the role of the lost LogScale node, follow these steps.

Create the humio-data directory and place a copy of the uuid file from the lost node — this file is present in the folder.

$ cp /backup/BACKUP-NAME/globaldata/cluster_membership-NODEID-UUID.uuid \
    /data/humio-data/cluster_membership.uuid

LogScale restores missing segment files when it discovers they are missing, and when they are present in the backup folder.