Hardware requirements depend on how much data will be ingested, and how many concurrent searches you will be running.

Whether deploying on physical hardware, virtual hardware on internal infrastructure (for example VMware), or cloud environments such as AWS or GCP, minimum recommended requirements:

If operating in Amazon Web Services in Elastic Compute nodes, try using the Ubuntu M5.4XL node type.

During deployment a location for the data that will be stored will be required, both for Kafka and LogScale nodes.

To create a new fileystem on your SSD or NVME storage, either the ext4 or xfs fileystem types are recommended:

shell> mkfs.ext4 /dev/nvme1n1p1

And an entry to /etc/fstab to ensure that the filesytem is mounted during boot.

For production deployments, authentication should be enabled. If authentication is not configured, LogScale runs in NO_AUTH mode, meaning that there are no access restrictions at all — anyone with access to the system can do anything. Refer to Authentication Configuration for different login options.

LogScale does not encrypt data on local drives as part of the application. Instead, administrators are encouraged to use relevant tooling for their operating system to fully encrypt the file systems holding LogScale and Kafka data.

A tool like cryptsetup can be used to encrypt an entire file system. Full Disk Encryption can be used to encrypt files at the hardware level.

When installing in a bare metal environment, disable the use of swap memory as it gives a false impression to Java and LogScale that more memory than physical memory is available.

To disable swap memory, remove any swap entries within the /etc/fstab.

For production usage, LogScale needs to be able to keep a lot of files open for sockets and actual files from the file system. The default limits on Unix systems are typically too low for any significant amount of data and concurrent users.

The limits can be verified using:

$ PID=`ps -ef | grep java | grep humio-assembly | head -n 1 | awk '{print $2}'`
$ cat /proc/$PID/limits | grep 'Max open files'

The minimum required settings depend on the number of open network connections and datasources. A value of at least 250,000 is recommended.

To modify the limits, create a file named 99-humio-limits.conf in the /etc/security/limits.d/ sub-directory. Copy these lines into that file:

# Raise limits for files:
humio soft nofile 250000
humio hard nofile 250000

Create another file with a text editor, this time in the /etc/pam.d/ sub-directory, and name it common-session. Copy these lines into it:

# Apply limits:
session required pam_limits.so

These settings apply to the next LogScale user login, not to any running processes.

Check the filesystem options on /tmp. LogScale makes use of the Facebook Zstandard real-time compression algorithm, which requires the ability to execute files directly from the configured temporary directory.

The options for the filesystem can be checked using mount:

$ mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,noexec,relatime,size=1967912k,nr_inodes=491978,mode=755,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=399508k,mode=755,inode64)
/dev/sda5 on / type ext4 (rw,relatime,errors=remount-ro)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noexec,seclabel)

The option can be temporarily removed noexec using mount to 'remount' the directory:

$ mount -oremount,exec /tmp

To permanently remove the noexec flag, update /etc/fstab to remove the flag from the options:

tmpfs /tmp tmpfs mode=1777,nosuid,nodev 0 0