Best Practice: Humio Installation using Ansible

Last Updated: 2022-07-21

This installation guide will describe how to install Humio on a cluster which consists of Ubuntu servers. For other Linux distributions you will need to adjust some parameters to fit your operating system. We recommend using Ubuntu to ensure that things run as expected.

In this guide we will use an automation tool called Ansible to simplify the installation of Humio. The guide is an example of how you would configure a Humio cluster using Ansible. It is not a production ready solution. For that you will need to configure load balancing and think about your disk layout among other things.

In this guide we provide an example of a configuration which contains both load balancing and S3 bucket storage: Humio installation - Ansible Bootstrap. The overall concepts in this guide are the same but infrastructure is set up using a tool called Terraform and it's specific to AWS.

Installation

The following components will be installed as part of the Humio system:

  • Humio Server (the core of the Humio system)

  • Java (a general-purpose, class-based, object-oriented programming language)

  • Kafka (a distributed event streaming platform)

  • Zookeeper (a centralized service for maintaining configuration information, naming and providing distributed synchronization. Used in conjunction with Kafka)

Prerequisites for Ansible to create the Humio solution
  • Create 3 Ubuntu servers in your environment (datacenter, in a virtual environment or AWS EC2 for testing purposes. If you decide to use AWS then be aware that we have an option to run on top of Kubernetes which is our recommended approach)

    Sizing can be determined by using this guide: Humio Sizing Bare-metal

  • Install Ansible on the instance which you use to provision the Humio solution. (If you re running on your own laptop you will install this software in the same way that you usually do it). On a server you might do something like this:

    shell
    sudo apt-get updatesudo apt-get upgrade -ysudo apt-get install -yq build-essential jq python3 openssh-client git iproute2 python3-pipsudo pip3 install ansible==2.9.13 boto3 boto-core
  • Clone the git repository to setup the basc Ansible environment:

    git clone https://github.com/humio/ansible-demo
  • Make sure that you have SSH Access to the servers/instances you want to install the Humio solution on (Bastions / SSM Works)

  • Working static inventory file (inventory.ini). By default you will have a file similar to:

    ini
    [humios]
    humio1 ansible_host=10.150.64.142 cluster_index=1 humio_network_interface=ens5
    humio2 ansible_host=10.150.70.243 cluster_index=2 humio_network_interface=ens5
    humio3 ansible_host=10.150.66.209 cluster_index=3 humio_network_interface=ens5
    
    [humios:vars]
    ansible_port=22
    ansible_user=ubuntu
    # ansible_ssh_private_key_file=/path/to/ssh_private_key_file
    
    [kafkas:children]
    humios
    
    [zookeepers:children]
    Humios

    You need to adjust the IP addresses to match the servers in your environment. The same goes for user name and key file.

  • Use ifconfig to determine network interface.

  • Verify that Ansible works with the parameters which are defined in the inventory file by running:

    ansible all -m ping -i inventory.ini

    Output should be similar to:

    humio1 | SUCCESS => {
    	"ansible_facts": {
     	"discovered_interpreter_python": "/usr/bin/python3"
    	},
    	"changed": false,
    	"ping": "pong"
    }
    humio2 | SUCCESS => {
    	"ansible_facts": {
     	"discovered_interpreter_python": "/usr/bin/python3"
    	},
    	"changed": false,
    	"ping": "pong"
    }
    humio3 | SUCCESS => {
    	"ansible_facts": {
     	"discovered_interpreter_python": "/usr/bin/python3"
    	},
    	"changed": false,
    	"ping": "pong"
    }
  • Before performing the actual Humio installation we will use Ansible Galaxy which is a package manager for Ansible to download the dependencies for the different roles we are installing (Humio, Java, Kafka and Zookeeper). This is done by running the following command:

    shell
    ansible-galaxy install -r requirements.yml --force-with-deps

    The result of running this command is that you end up with an Ansible playbook called site.yaml. This contains the steps that Ansible needs to run to set up the necessary roles for Humio to work (Humio, Java, Kafka and Zookeeper).

You can now use the Ansible playbook and install Humio by running the following command:

shell
ansible-playbook site.yml
Verify installation

To check that the installation was successful you can run the following command:

curl http://10.150.64.142:8080/api/v1/status

You should see something like this:

{"status": "OK", "version": "1.38.1--build-171452--sha-48e13926a69bf50e1f7a1b172167141fdeedd977"}

Running the following command:

sudo systemctl status humio@0

Should return something like:

humio@0.service - Humio service
 Loaded: loaded (/etc/systemd/system/humio@.service; enabled; vendor preset: enabled)
 Drop-In: /etc/systemd/system/humio@0.s ervice.d
 numa.conf
 Active: active (running) since Tue 2022-05-10 23:36:53 UTC; 6min ago