Deploying a LogScale Cluster using Containers
For production purposes LogScale is distributed as the
humio-core Docker image that
does not include Kafka.
To deploy a LogScale cluster using the Kafka cluster configured in Deploying a Kafka Cluster using Containers:
Create an empty file on the host machine to store the LogScale configuration. For example,
logscale.env.Add the following lines to the configuration file:
These lines configure the following settings:
AUTHENTICATION_METHOD=single-userSets the authentication method. Initially a single-user model is used. The
SINGLE_USER_USERNAMEandSINGLE_USER_PASSWORDcan be added to set a username and password to connect to the cluster. For information on other authorization methods, see Authentication and Identity Providers.Sets the default port that will be used to access LogScale.
Sets the port for ingesting data using the Elastic bulk protocol.
KAFKA_SERVERS=kafka1:9092,kafka2:9092,kafka3:9092Sets the list of Kafka servers required by LogScale.
EXTERNAL_URL=http://127.0.0.1:8080Sets the external URL that LogScale will listen for connections. The above uses the localhost address (127.0.0.1) but this should be changed to the hostname of the LogScale node, for example
logscale1. Each node in the LogScale will have a different external URL.Sets the URL of the main access point of the LogScale cluster. The example uses the localhost address (127.0.0.1), but this should be changed to the IP address of the load balancer used to access the cluster. See Installing Load Balancers for more information.
LogScale refers to these configurations as the LogScale environment variables.
For more information on each of these environment variables, see the Configuration Variables.
To make changes to the settings in the environment file, simply stopping and starting the container will not work. Stop the container using docker rm the container and docker run it again to pick up any changes.
Create an empty directory on the host machine to store data for LogScale, for example:
shell$mkdir /dataDownload (pull) the latest LogScale image:
shell$sudo docker pull humio/humio-coreOr if on the Arm platform, specify required version and platform tags, for example:
shell$sudo docker pull humio/humio-core:1.189.1--arm64The latest version of the image can be found on Docker Hub.
Run the LogScale Docker image as a container
shell$sudo docker run -d --restart always --net=host \ -v /data:/data \ --stop-timeout 300 \ --env-file $PATH_TO_CONFIG_FILE --name humio-core humio/humio-coreReplace
$PATH_TO_CONFIG_FILEwith the path of the configuration file you created.
This configuration will need to be repeated for each node in the LogScale cluster. Note that node IDs for LogScale nodes are allocated automatically.
Verify that LogScale is able to start using the configuration provided by looking at the log file. In particular, it should not keep logging problems connecting to Kafka.
$ grep 'LogScale server is now running!' /data/logs/humio_std_out.log
$ grep -i 'kafka' /data/logs/humio_std_out.log
LogScale is now running. Navigate to
http://localhost:8080 to view the LogScale Web UI.
In the above example, you started the LogScale container with full access to the network of the host machine. In a production environment, you should restrict this access by using a firewall, or adjusting the Docker network configuration.
If you experience any problems, first look at Additional Considerations.
Data and Log Directories
Using the above configuration and scripts you'll have a
data directory with various subdirectories,
including humio-data, and
logs. If you're running the
humio/single-node-demo,
you'll also have kafka-data, and
zookeeper-data.
While retaining the default locations are recommended, they can be
changed. See the following variables for more information:
DIRECTORY, HUMIO_AUDITLOG_DIR,
HUMIO_DEBUGLOG_DIR, and JVM_LOG_DIR. It
is also possible to change the Java temporary directory using
JVM_TMP_DIR.
Starting a LogScale Container Deployment at Boot as a Service
There are different ways of starting the Docker container as a service. In the above example, we used Docker's restart policies. LogScale can be started using a process manager.
If you receive this warning after starting up the LogScale service, please ignore it. This does not affect the LogScale service.
LogScale server is now running!
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.humio.util.FileUtilsJNA (file:/app/humio/humio-assembly.jar) to field sun.nio.ch.FileChannelImpl.fd
WARNING: Please consider reporting this to the maintainers of com.humio.util.FileUtilsJNA
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release.