Deploying a LogScale Cluster using Containers
LogScale is distributed as a core Docker image that contains only
LogScale; use the humio/humio-core edition for
distributed deployments.
To deploy a LogScale cluster using the Kafka cluster configured in Deploying a Kafka Cluster using Containers:
Create an empty file on the host machine to store the LogScale configuration. For example,
humio.conf.Add the following lines to the configuration file:
iniAUTHENTICATION_METHOD=single-userDIRECTORY=/var/humio/dataHUMIO_AUDITLOG_DIR=/var/log/humioHUMIO_DEBUGLOG_DIR=/var/log/humioJVM_LOG_DIR=/var/log/humioHUMIO_PORT=8080ELASTIC_PORT=9200ZOOKEEPER_URL=kafka1:2181,kafka2:2181,kafka3:2181KAFKA_SERVERS=kafka1:9092,kafka2:9093,kafka3:9094EXTERNAL_URL=http://127.0.0.1:8080PUBLIC_URL=http://127.0.0.1These lines configure the following settings:
AUTHENTICATION_METHOD=single-userSets the authentication method. Initially a single-user model is used. The
SINGLE_USER_USERNAMEandSINGLE_USER_PASSWORDcan be added to set a username and password to connect to the cluster. For information on other authorization methods, see Authentication & Identity Providers.Sets the directory where LogScale will store data.
HUMIO_AUDITLOG_DIR=/var/log/humioSets the directory for logs from LogScale for auditing.
HUMIO_DEBUGLOG_DIR=/var/log/humioSets the directory for logs from LogScale.
JVM_LOG_DIR=/var/log/humioSets the directory for logs from LogScale.
Sets the default port that will be used to access LogScale.
Sets the port for ingesting data using the Elastic bulk protocol.
ZOOKEEPER_URL=kafka1:2181,kafka2:2181,kafka3:2181Sets the list of hosts where ZooKeeper is installed.
KAFKA_SERVERS=kafka1:9092,kafka2:9092,kafka3:9092Sets the list of Kafka servers required by LogScale.
EXTERNAL_URL=http://127.0.0.1:8080Sets the external URL that LogScale will listen for connections. The above uses the localhost address (127.0.0.1) but this should be changed to the hostname of the LogScale node, for example
logscale1.Sets the public URL that LogScale will use to advertise and create links. The above uses the localhost address (127.0.0.1), but this should be changed to the IP address of the load balance used to access the cluster. See Installing Load Balancers for more information.
LogScale refers to these configurations as the LogScale environment variables. The lines in the above configure the following:
For more information on each of these environment variables, see the LogScale Configuration Parameters.
To make changes to the settings in the environment file, simply stopping and starting the container will not work. Stop the container using docker rm the container and docker run it again to pick up changes.
Create an empty directory on the host machine to store data for LogScale
shell$mkdir /data/humio-dataDownload (pull) the latest LogScale image:
shell$docker pull humio/humio-coreRun the LogScale Docker image as a container
shell$docker run -d --restart always --net=host \ -v /data/logs:/data/logs \ -v /data/humio-data:/data/humio-data \ --stop-timeout 300 \ --env-file $PATH_TO_CONFIG_FILE --name humio-core humio/humio-coreReplace
/data/humio-databefore the:with the path to the data directory and$PATH_TO_CONFIG_FILEwith the path of the configuration file you created.
This configuration will need to be repeated for each node in the LogScale cluster.
Verify that LogScale is able to start using the configuration provided by looking at the log file. In particular, it should not keep logging problems connecting to Kafka.
$ grep 'LogScale server is now running!' /data/logs/humio_std_out.log
$ grep -i 'kafka' /data/logs/humio_std_out.log
LogScale is now running. Navigate to http://localhost:8080 to
view the LogScale Web UI.
In the above example, we started the LogScale container with full access to the network of the host machine. In a production environment, you should restrict this access by using a firewall, or adjusting the Docker network configuration.
Starting a LogScale Container Deployment at Boot as a Service
There are different ways of starting the Docker container as a service. In the above example, we used Docker's restart policies. LogScale can be started using a process manager.
If you receive this warning after starting up the LogScale service, please ignore it. This does not affect the LogScale service.
LogScale server is now running!
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.humio.util.FileUtilsJNA (file:/app/humio/humio-assembly.jar) to field sun.nio.ch.FileChannelImpl.fd
WARNING: Please consider reporting this to the maintainers of com.humio.util.FileUtilsJNA
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release.