Deploying a LogScale Cluster using Containers
LogScale is distributed as a core Docker image that contains only
LogScale; use the
humio/humio-core
edition for
distributed deployments.
To deploy a LogScale cluster using the Kafka cluster configured in Deploying a Kafka Cluster using Containers:
Create an empty file on the host machine to store the LogScale configuration. For example,
humio.conf
.Add the following lines to the configuration file:
iniAUTHENTICATION_METHOD=single-user
DIRECTORY=/var/humio/data
HUMIO_AUDITLOG_DIR=/var/log/humio
HUMIO_DEBUGLOG_DIR=/var/log/humio
JVM_LOG_DIR=/var/log/humio
HUMIO_PORT=8080
ELASTIC_PORT=9200
ZOOKEEPER_URL=kafka1:2181,kafka2:2181,kafka3:2181
KAFKA_SERVERS=kafka1:9092,kafka2:9093,kafka3:9094
EXTERNAL_URL=http://127.0.0.1:8080
PUBLIC_URL=http://127.0.0.1
These lines configure the following settings:
AUTHENTICATION_METHOD=single-user
Sets the authentication method. Initially a single-user model is used. The
SINGLE_USER_USERNAME
andSINGLE_USER_PASSWORD
can be added to set a username and password to connect to the cluster. For information on other authorization methods, see Authentication & Identity Providers.Sets the directory where LogScale will store data.
HUMIO_AUDITLOG_DIR=/var/log/humio
Sets the directory for logs from LogScale for auditing.
HUMIO_DEBUGLOG_DIR=/var/log/humio
Sets the directory for logs from LogScale.
JVM_LOG_DIR=/var/log/humio
Sets the directory for logs from LogScale.
Sets the default port that will be used to access LogScale.
Sets the port for ingesting data using the Elastic bulk protocol.
ZOOKEEPER_URL=kafka1:2181,kafka2:2181,kafka3:2181
Sets the list of hosts where ZooKeeper is installed.
KAFKA_SERVERS=kafka1:9092,kafka2:9092,kafka3:9092
Sets the list of Kafka servers required by LogScale.
EXTERNAL_URL=http://127.0.0.1:8080
Sets the external URL that LogScale will listen for connections. The above uses the localhost address (127.0.0.1) but this should be changed to the hostname of the LogScale node, for example
logscale1
.Sets the public URL that LogScale will use to advertise and create links. The above uses the localhost address (127.0.0.1), but this should be changed to the IP address of the load balance used to access the cluster. See Installing Load Balancers for more information.
LogScale refers to these configurations as the LogScale environment variables. The lines in the above configure the following:
For more information on each of these environment variables, see the LogScale Configuration Parameters.
To make changes to the settings in the environment file, simply stopping and starting the container will not work. Stop the container using docker rm the container and docker run it again to pick up changes.
Create an empty directory on the host machine to store data for LogScale
shell$
mkdir /data/humio-data
Download (pull) the latest LogScale image:
shell$
docker pull humio/humio-core
Run the LogScale Docker image as a container
shell$
docker run -d --restart always --net=host \ -v /data/logs:/data/logs \ -v /data/humio-data:/data/humio-data \ --stop-timeout 300 \ --env-file $PATH_TO_CONFIG_FILE --name humio-core humio/humio-core
Replace
/data/humio-data
before the:
with the path to the data directory and$PATH_TO_CONFIG_FILE
with the path of the configuration file you created.
This configuration will need to be repeated for each node in the LogScale cluster.
Verify that LogScale is able to start using the configuration provided by looking at the log file. In particular, it should not keep logging problems connecting to Kafka.
$ grep 'LogScale server is now running!' /data/logs/humio_std_out.log
$ grep -i 'kafka' /data/logs/humio_std_out.log
LogScale is now running. Navigate to http://localhost:8080
to
view the LogScale Web UI.
In the above example, we started the LogScale container with full access to the network of the host machine. In a production environment, you should restrict this access by using a firewall, or adjusting the Docker network configuration.
Starting a LogScale Container Deployment at Boot as a Service
There are different ways of starting the Docker container as a service. In the above example, we used Docker's restart policies. LogScale can be started using a process manager.
If you receive this warning after starting up the LogScale service, please ignore it. This does not affect the LogScale service.
LogScale server is now running!
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.humio.util.FileUtilsJNA (file:/app/humio/humio-assembly.jar) to field sun.nio.ch.FileChannelImpl.fd
WARNING: Please consider reporting this to the maintainers of com.humio.util.FileUtilsJNA
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release.