LogScale on Bare Metal - Installation Preparation

To prepare for installation of LogScale, there must be a suitable environment to execute the LogScale software.

Hardware Requirements

Hardware requirements depend on how much data will be ingested, and how many concurrent searches you will be running.

Whether deploying on physical hardware, virtual hardware on internal infrastructure (for example VMware), or cloud environments such as AWS or GCP, minimum recommended requirements:

  • 16 CPU cores

  • 16 GB RAM

  • 1 Gbps network interface

  • SSD- or NVME-based storage

If operating in Amazon Web Services in Elastic Compute nodes, try using the Ubuntu M5.4XL node type.

Kernel Configuration

Ensure the Linux kernel is configured with the following settings:

Kernel Parameter Recommended Value Description
net.core.rmem_default 10000000 Defines the default size of the receive buffer (in bytes) used by sockets. This buffer is used to store incoming data before it is processed by the application. The receive buffer is crucial for network performance, as it determines how much data can be queued for a socket before the application processes it. If the buffer is too small, packets may be dropped due to insufficient memory, especially under high network load.
net.core.wmem_default 10000000 Specifies the default size of the socket send buffer (write buffer) used by network sockets. This buffer temporarily holds outgoing data before it is sent to the network. The parameter is part of the networking stack and can be configured using the sysctl interface.
net.core.rmem_max 16777216 Defines the maximum size of the receive buffer for a socket, in bytes. This parameter is part of the networking stack and can be adjusted to optimize network performance, particularly in high-throughput or high-latency scenarios.
net.core.wmem_max 16777216 Specifies the maximum size of the send buffer (in bytes) that can be allocated for a socket. It is part of the system's networking configuration and is used to control the maximum amount of memory that a socket's write buffer can use.
net.ipv4.tcp_rmem 4096, 87380, 16777216 Configures the TCP receive buffer sizes for sockets. This parameter is part of the networking stack and is used to optimize TCP performance by controlling how much memory is allocated for the TCP receive buffer. It is represented as an array of three integers: minimum size, default size, and maximum size.
net.ipv4.tcp_wmem 4096, 65536, 16777216 Specifies the memory limits for TCP socket send buffers. It is part of the networking stack configuration and controls the minimum, default, and maximum memory allocated for TCP transmit buffers.
net.core.netdev_max_backlog 30000 Specifies the maximum number of packets that can be queued on the network interface's backlog queue before being processed by the kernel's networking subsystem.
Setting the Parameters

To get and temporarily set net.core.rmem_default:

shell
# Get the default size of the receive buffer (in bytes) used by sockets
$ sysctl net.core.rmem_default
# Set to 10000000 bytes.
$ sudo sysctl -w net.core.rmem_default=10000000

To get and temporarily set net.core.wmem_default:

shell
# Get the default size of the socket send (write) buffer used by sockets
$ sysctl net.core.wmem_default
# Set to 10000000 bytes.
$ sudo sysctl -w net.core.wmem_default=10000000

To get and temporarily set net.core.rmem_max:

shell
# Get the maximum size of the receive buffer for a socket, in bytes.
$ sysctl net.core.rmem_max
# Set to 16777216 bytes.
$ sudo sysctl -w net.core.rmem_max=16777216

To get and temporarily set net.core.wmem_max:

shell
# Get the maximum size of the send (write) buffer (in bytes) that can be allocated for a socket.
$ sysctl net.core.wmem_max
# Set to 16777216 bytes.
$ sudo sysctl -w net.core.wmem_max=16777216

To get and temporarily set net.ipv4.tcp_rmem:

shell
# Get the TCP receive buffer sizes for sockets.
$ sysctl net.ipv4.tcp_rmem
# Set minimum, default, and maximum buffer size.
$ sudo sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"

To get and temporarily set net.ipv4.tcp_wmem:

Configures the TCP receive buffer sizes for sockets.
shell
# Get the TCP send buffer sizes for sockets.
$ sysctl net.ipv4.tcp_wmem
# Set minimum, default, and maximum buffer size.
$ sudo sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"

To get and temporarily set net.core.netdev_max_backlog:

shell
# Get the maximum number of packets that can be queued on the network interface's backlog queue before being processed by the kernel's networking subsystem
$ sysctl net.core.netdev_max_backlog
# Set to 30000 packets.
$ sudo sysctl -w net.core.netdev_max_backlog=30000

Note

To set parameters so they are persisted, you need to update the appropriate configuration file, such as /etc/sysctl.conf or create a file in /etc/sysctl.d. Then run a command such as the following for each parameter:

shell
# Write configuration to file
$ echo "net.ipv4.tcp_wmem=4096 65536 16777216" >> /etc/sysctl.conf
# Reload sysctl config file
$ sysctl -p

File System

During deployment a location for the data that will be stored will be required, both for Kafka and LogScale nodes.

To create a new fileystem on your SSD or NVME storage, either the ext4 or xfs fileystem types are recommended:

shell
shell> mkfs.ext4 /dev/nvme1n1p1

And an entry to /etc/fstab to ensure that the filesytem is mounted during boot.

Enable Authentication

For production deployments, authentication should be enabled. If authentication is not configured, LogScale runs in NO_AUTH mode, meaning that there are no access restrictions at all — anyone with access to the system can do anything. Refer to Authentication Configuration for different login options.

Encryption for Local Storage

LogScale does not encrypt data on local drives as part of the application. Instead, administrators are encouraged to use relevant tooling for their operating system to fully encrypt the file systems holding LogScale and Kafka data.

A tool like cryptsetup can be used to encrypt an entire file system. Full Disk Encryption can be used to encrypt files at the hardware level.

Disable Swap Memory on Bare-metal

When installing in a bare metal environment, disable the use of swap memory as it gives a false impression to Java and LogScale that more memory than physical memory is available.

To disable swap memory, remove any swap entries within the /etc/fstab.

Increase Open File Limit

For production usage, LogScale needs to be able to keep a lot of files open for sockets and actual files from the file system. The default limits on Unix systems are typically too low for any significant amount of data and concurrent users.

The limits can be verified using:

shell
$ PID=`ps -ef | grep java | grep humio-assembly | head -n 1 | awk '{print $2}'`
$ cat /proc/$PID/limits | grep 'Max open files'

The minimum required settings depend on the number of open network connections and datasources. A value of at least 250,000 is recommended.

To modify the limits, create a file named 99-humio-limits.conf in the /etc/security/limits.d/ sub-directory. Copy these lines into that file:

ini
# Raise limits for files:
humio soft nofile 250000
humio hard nofile 250000

Create another file with a text editor, this time in the /etc/pam.d/ sub-directory, and name it common-session. Copy these lines into it:

ini
# Apply limits:
session required pam_limits.so

These settings apply to the next LogScale user login, not to any running processes.

Check noexec on /tmp

Check the filesystem options on /tmp. LogScale makes use of the Facebook Zstandard real-time compression algorithm, which requires the ability to execute files directly from the configured temporary directory.

The options for the filesystem can be checked using mount:

shell
$ mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,noexec,relatime,size=1967912k,nr_inodes=491978,mode=755,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=399508k,mode=755,inode64)
/dev/sda5 on / type ext4 (rw,relatime,errors=remount-ro)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noexec,seclabel)

The option can be temporarily removed noexec using mount to 'remount' the directory:

shell
$ mount -oremount,exec /tmp

To permanently remove the noexec flag, update /etc/fstab to remove the flag from the options:

shell
tmpfs /tmp tmpfs mode=1777,nosuid,nodev 0 0