Humio has a built-in backup facility. It only requires a separate directory to which Humio will write; preferably, on another disk or network drive separate from the data directory. When configured, Humio keeps a full backup of the current state in this directory. A Humio node can start with no data and restore the previous state from such a backup.
The files written on the backup drive are encrypted using a secret provided during Humio's configuration, and allow you to store the backup on a network drive where others may have read access.
Third Party Backup Software
Humio is designed to support solutions like Amazon S3 for archiving. This is not a full backup, but will archive the data so that it can be re-ingested by another or the same Humio server at some later date.
You can do a full backup using any backup software that is able to back up all the files in the Humio data directory. Note that the software needs to support "sparse files" to be efficient.
You can back up your Humio installation by adding a special mounted directory when you run the Docker container. Humio writes its backup files to this directory. Each Humio node can have its own backup directory that is not shared with the other nodes. This is possible since it is not always feasible to create one backup directory that can hold all the data in a Humio cluster. In the backup directory, each Humio node will create a subdirectory with its node ID as name and use this directory.
First, create an empty directory on the host machine to store data for Humio. You would do that like so:
We recommend creating the backup directory on a different disk from the main Humio data directory. Make the directory a mount point for a network drive or other similar separation from the main data drive.
Next, edit the Humio configuration file to set the backup parameters. Add these lines to that file:
Humio encrypts all backups with a secret key that you provide. This means that you can safely store backups on an unencrypted disk, or send them over the Internet. Keep the secret key safe and store it in another place. You cannot recover the backup if you lose access to it. If you lose the secret, delete all the files in the backup, or provide a new location to backup to, and start over. Humio will then write a fresh backup.
The next step is to run Humio using the Docker run command with the following argument like so:
run -v /humio-backups-on-host:/backup
This maps the backups directory on the host. In this example,
/humio-backups-on-host will send the backup to the
/backup directory in the container. Humio will then
start backing up data to the specified directory.
The procedure is similar. Instead of mounting the directory using -v, you specify the location using BACKUP_DIR. A full example configuration is as follows:
BACKUP_NAME=humio-backup BACKUP_KEY=mysecretkey-myhost-+R+q(AB9QG86xZMCKGyj BACKUP_DIR=/mnt/my-net-server/humio-backup01
Deleting Data in Backup
When is data deleted in backup? Right when retention kicks in and
deletes data in Humio. It is possible to configure a delay so that data
is not deleted in the backup until some time has elapsed since it was
deleted in Humio. This is configured using
DELETE_BACKUP_AFTER_MILLIS. By default Humio will not
delete data in the backup until seven days after the data was deleted in
Restoring a Backup
Humio can restore all events that were stored in segment files for a
Humio node in the
humio-data dir from the backup. It
can also help a node claim the nodeID of a lost node in a cluster.
If your node is lost, but you have reinstalled the OS on the existing hardware, or perhaps found a spare server that will now take on the role of the lost Humio node, follow these steps.
humio-data directory and place a copy of
the uuid file from the lost node — this file is present in the
cp /backup/BACKUP-NAME/globaldata/cluster_membership-NODEID-UUID.uuid \ /data/humio-data/cluster_membership.uuid
Humio restores missing segment files when it discovers they are missing, and when they are present in the backup folder.