How-To: Configuring a Standalone Installation to Start at Boot

Important

This article no longer matches the current deployment and installation model for LogScale. It is kept here for reference purposes only.

If you have installed a standalone instance of LogScale and want to be able to ensure that LogScale starts automatically on boot. You will need to configure three separate boot scripts that are designed to operate in order to ensure that the correct services are available. The three services are:

  • ZooKeeper (up to LogScale 1.107)

  • Kafka

  • LogScale

Kafka relies on ZooKeeper running; LogScale relies on Kafka and ZooKeeper running (up to LogScale 1.107). By configuring the startup scripts to rely on each other it ensures that all the services come up in the correct order, and each service starts up only when the service is available.

The systemd service configures services to start at boot and manages the services ensuring that they continue to execute.

Configuring ZooKeeper

To configure ZooKeeper for systemd, create a boot script configuration in /etc/systemd/system/zookeeper.service. The exact location of your ZooKeeper installation may differ from the example below.

ini
[Unit]

Description=Apache ZooKeeper Server
Requires=network.target remote-fs.target
After=network.target remote-fs.target

[Service]

Type=simple
ExecStart=/usr/local/kafka-server/kafka_2.13-2.8.0/bin/zookeeper-server-start.sh /usr/local/kafka-server/kafka_2.13-2.8.0/config/zookeeper.properties
ExecStop=/usr/local/kafka-server/kafka_2.13-2.8.0/bin/zookeeper-server-stop.sh
Restart=on-abnormal

[Install]

WantedBy=multi-user.target

Once the file has been created, start the service using systemctl:

shell
$ systemctl start zookeeper

Now enable the service to be started on boot: 

$ systemctl enable zookeeper

Configuring Kafka

To configure Kafka for systemd, create a boot script configuration in /etc/systemd/system/kafka.service. The exact location of your Kafka installation may differ from the example below.

ini
[Unit]

Description=Apache Kafka Server
Documentation=http://kafka.apache.org/documentation.html
Requires=zookeeper.service
After=zookeeper.service

[Service]

Type=simple
Environment="JAVA_HOME=/usr/lib/jvm/java-13-openjdk-amd64"
ExecStart=/usr/local/kafka-server/kafka_2.13-2.8.0/bin/kafka-server-start.sh /usr/local/kafka-server/kafka_2.13-2.8.0/config/server.properties
ExecStop=/usr/local/kafka-server/kafka_2.13-2.8.0/bin/kafka-server-stop.sh
Restart=on-abnormal

[Install]

WantedBy=multi-user.target

Once the file has been created, start the service using systemctl:

shell
$ systemctl start kafka

Now enable the service to be started on boot:

shell
$ systemctl enable kafka

Configuring LogScale

To configure LogScale for systemd, create a boot script configuration in /etc/systemd/system/kafka.service. The exact location of your Kafka installation may differ from the example below.

ini
[Unit]

Description=LogScale service
After=network.service
Requires=kafka.service

[Service]

Type=notify
Restart=on-abnormal
User=humio
Group=humio
LimitNOFILE=250000:250000
EnvironmentFile=/etc/humio/server.conf
WorkingDirectory=/var/humio

ExecStart=/usr/bin/java -server -XX:+UseParallelOldGC -Xms4G -Xmx32G -XX:MaxDirectMemorySize=8G -Xss2M --add-exports java.base/jdk.internal.util=ALL-UNNAMED -XX:CompileCommand=dontinline,com/humio/util/HotspotUtilsJ.dontInline -Xlog:gc*,gc+jni=debug:file=/var/log/humio/gc_humio.log:time,tags:filecount=5,filesize=102400 -Dhumio.auditlog.dir=/var/log/humio -Dhumio.debuglog.dir=/var/log/humio -jar /opt/humio/server.jar

[Install]

WantedBy=default.target

Once the file has been created, start the service using systemctl:

shell
$ systemctl start humio

Now enable the service to be started on boot:

shell
$ systemctl enable humio

Checking Running State

You can obtain information about the current status of a managed service using the systemctl status command:

shell
$ systemctl status humio

This will output the status information and the most recent journal messages:

default
humio.service - LogScale service
     Loaded: loaded (/etc/systemd/system/humio.service; disabled; vendor preset: enabled)
     Active: active (running) since Mon 2021-09-06 18:30:01 BST; 7h ago
   Main PID: 2022 (java)
      Tasks: 86 (limit: 4612)
     Memory: 2.3G
     CGroup: /system.slice/humio.service
             2022 /usr/bin/java -server -XX:+UseParallelOldGC -Xms2G -Xmx2G -XX:MaxDirectMemorySize=4G -Xss2M --add-exports java.base/jdk.internal.util=ALL-UNNAMED -XX:CompileCommand=dontinline,com/humio/util/HotspotUtilsJ.dontInline -Xlog:gc*,gc+jni=debug:file=/var/log/humio/gc_humio.log:time,tags:filecount=5,filesize=102400 -Dhumio.auditlog.dir=/var/log/humio -Dhumio.debuglog.dir=/var/log/humio -jar /opt/humio/server.jar

Sep 06 18:30:01 humiobase java[2022]: LogScale server is now running!
Sep 06 18:30:01 humiobase systemd[1]: Started LogScale service.
Sep 06 18:30:06 humiobase java[2022]: /usr/bin/lscpu
Sep 06 19:30:06 humiobase java[2022]: /usr/bin/lscpu
Sep 06 20:30:06 humiobase java[2022]: /usr/bin/lscpu
Sep 06 21:30:06 humiobase java[2022]: /usr/bin/lscpu
Sep 06 22:30:06 humiobase java[2022]: /usr/bin/lscpu
Sep 06 23:30:06 humiobase java[2022]: /usr/bin/lscpu
Sep 07 00:30:06 humiobase java[2022]: /usr/bin/lscpu
Sep 07 01:30:06 humiobase java[2022]: /usr/bin/lscpu

Stopping Services

You can stop any service managed by systemd by using the systemctl stop command:

shell
$ systemctl stop humio

Nore that if you stop a service which other services rely on, all corresponding services will be stopped

Checking Errors

The journalctl command will show you the log output from running services. In the event of a startup error, run:

shell
$ journalctl -xe

This will output the journal log of the systemd service which will show any key messages including the most recent failure. For example:

default
Sep 07 01:34:10 humiobase systemd[1]: Started Session 16 of user mc.
-- Subject: A start job for unit session-16.scope has finished successfully
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- A start job for unit session-16.scope has finished successfully.
--
-- The job identifier is 2783.
Sep 07 01:35:06 humiobase zookeeper-server-start.sh[839]: [2021-09-07 01:35:06,861] INFO Processing mntr command from /127.0.0.1:48416 (org.apache.zookeeper.server.NIOServerC>
Sep 07 01:35:06 humiobase zookeeper-server-start.sh[839]: [2021-09-07 01:35:06,861] INFO Processing stat command from /127.0.0.1:48418 (org.apache.zookeeper.server.NIOServerC>
Sep 07 01:35:06 humiobase zookeeper-server-start.sh[839]: [2021-09-07 01:35:06,861] INFO Stat command output (org.apache.zookeeper.server.command.AbstractFourLetterCommand)
Sep 07 01:36:06 humiobase zookeeper-server-start.sh[839]: [2021-09-07 01:36:06,863] INFO Processing mntr command from /127.0.0.1:48446 (org.apache.zookeeper.server.NIOServerC>
Sep 07 01:36:06 humiobase zookeeper-server-start.sh[839]: [2021-09-07 01:36:06,863] INFO Processing stat command from /127.0.0.1:48448 (org.apache.zookeeper.server.NIOServerC>
Sep 07 01:36:06 humiobase zookeeper-server-start.sh[839]: [2021-09-07 01:36:06,863] INFO Stat command output (org.apache.zookeeper.server.command.AbstractFourLetterCommand)