Skip to content
LogScale Self-Hosted Deployment
This is an archived version of the documentation downloaded from LogScale Documentation Archives. Up to date documentation, and documentation for later versions, is available on LogScale Documentation

## LogScale on Bare Metal - Installation Preparation

To prepare for installation of LogScale, there must be a suitable environment to execute the LogScale software.

### Hardware Requirements

Hardware requirements depend on how much data will be ingested, and how many concurrent searches you will be running.

Whether deploying on physical hardware, virtual hardware on internal infrastructure (for example VMware), or cloud environments such as AWS or GCP, minimum recommended requirements:

- 16 CPU cores

- 16 GB RAM

- 1 Gbps network interface

- SSD- or NVME-based storage

If operating in Amazon Web Services in Elastic Compute nodes, try using the Ubuntu M5.4XL node type.

### Kernel Configuration

Ensure the Linux kernel is configured with the following settings:

Kernel Parameter | Recommended Value | Description
--- | --- | ---
`net.core.rmem_default` | 10000000 | Defines the default size of the receive buffer (in bytes) used by sockets. This buffer is used to store incoming data before it is processed by the application. The receive buffer is crucial for network performance, as it determines how much data can be queued for a socket before the application processes it. If the buffer is too small, packets may be dropped due to insufficient memory, especially under high network load.
`net.core.wmem_default` | 10000000 | Specifies the default size of the socket send buffer (write buffer) used by network sockets. This buffer temporarily holds outgoing data before it is sent to the network. The parameter is part of the networking stack and can be configured using the `sysctl` interface.
`net.core.rmem_max` | 16777216 | Defines the maximum size of the receive buffer for a socket, in bytes. This parameter is part of the networking stack and can be adjusted to optimize network performance, particularly in high-throughput or high-latency scenarios.
`net.core.wmem_max` | 16777216 | Specifies the maximum size of the send buffer (in bytes) that can be allocated for a socket. It is part of the system's networking configuration and is used to control the maximum amount of memory that a socket's write buffer can use.
`net.ipv4.tcp_rmem` | 4096, 87380, 16777216 | Configures the TCP receive buffer sizes for sockets. This parameter is part of the networking stack and is used to optimize TCP performance by controlling how much memory is allocated for the TCP receive buffer. It is represented as an array of three integers: minimum size, default size, and maximum size.
`net.ipv4.tcp_wmem` | 4096, 65536, 16777216 | Specifies the memory limits for TCP socket send buffers. It is part of the networking stack configuration and controls the minimum, default, and maximum memory allocated for TCP transmit buffers.
`net.core.netdev_max_backlog` | 30000 | Specifies the maximum number of packets that can be queued on the network interface's backlog queue before being processed by the kernel's networking subsystem.

#### Setting the Parameters

To get and temporarily set `net.core.rmem_default`:

shell

    # Get the default size of the receive buffer (in bytes) used by sockets
    $ sysctl net.core.rmem_default
    # Set to 10000000 bytes.
    $ sudo sysctl -w net.core.rmem_default=10000000

To get and temporarily set `net.core.wmem_default`:

shell

    # Get the default size of the socket send (write) buffer used by sockets
    $ sysctl net.core.wmem_default
    # Set to 10000000 bytes.
    $ sudo sysctl -w net.core.wmem_default=10000000

To get and temporarily set `net.core.rmem_max`:

shell

    # Get the maximum size of the receive buffer for a socket, in bytes.
    $ sysctl net.core.rmem_max
    # Set to 16777216 bytes.
    $ sudo sysctl -w net.core.rmem_max=16777216

To get and temporarily set `net.core.wmem_max`:

shell

    # Get the maximum size of the send (write) buffer (in bytes) that can be allocated for a socket.
    $ sysctl net.core.wmem_max
    # Set to 16777216 bytes.
    $ sudo sysctl -w net.core.wmem_max=16777216

To get and temporarily set `net.ipv4.tcp_rmem`:

shell

    # Get the TCP receive buffer sizes for sockets.
    $ sysctl net.ipv4.tcp_rmem
    # Set minimum, default, and maximum buffer size.
    $ sudo sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"

To get and temporarily set `net.ipv4.tcp_wmem`:

shell

    # Get the TCP send buffer sizes for sockets.
    $ sysctl net.ipv4.tcp_wmem
    # Set minimum, default, and maximum buffer size.
    $ sudo sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"

To get and temporarily set `net.core.netdev_max_backlog`:

shell

    # Get the maximum number of packets that can be queued on the network interface's backlog queue before being processed by the kernel's networking subsystem
    $ sysctl net.core.netdev_max_backlog
    # Set to 30000 packets.
    $ sudo sysctl -w net.core.netdev_max_backlog=30000

### Note

To set parameters so they are persisted, you need to update the appropriate configuration file, such as `/etc/sysctl.conf` or create a file in `/etc/sysctl.d`. Then run a command such as the following for each parameter:

shell

    # Write configuration to file
    $ echo "net.ipv4.tcp_wmem=4096 65536 16777216" >> /etc/sysctl.conf
    # Reload sysctl config file
    $ sysctl -p

### File System

During deployment a location for the data that will be stored will be required, both for Kafka and LogScale nodes.

To create a new filesystem on your SSD or NVME storage, either the `ext4` or `xfs` filesystem types are recommended:

shell

    shell> mkfs.ext4 /dev/nvme1n1p1

And an entry to `/etc/fstab` to ensure that the filesytem is mounted during boot.

### Enable Authentication

For production deployments, authentication should be enabled. If authentication is not configured, LogScale runs in `NO_AUTH` mode, meaning that there are no access restrictions at all — anyone with access to the system can do anything. Refer to Configure Security for different login options.

### Encryption for Local Storage

LogScale does not encrypt data on local drives as part of the application. Instead, administrators are encouraged to use relevant tooling for their operating system to fully encrypt the file systems holding LogScale and Kafka data.

A tool like cryptsetup can be used to encrypt an entire file system. Full Disk Encryption can be used to encrypt files at the hardware level.

### Disable Swap Memory on Bare-metal

When installing in a bare metal environment, disable the use of swap memory as it gives a false impression to Java and LogScale that more memory than physical memory is available.

To disable swap memory, remove any swap entries within the `/etc/fstab`.

### Check `noexec` on `/tmp`

Check the filesystem options on `/tmp`. LogScale makes use of the Facebook Zstandard real-time compression algorithm, which requires the ability to execute files directly from the configured temporary directory.

The options for the filesystem can be checked using mount:

shell

    $ mount
    sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
    proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
    udev on /dev type devtmpfs (rw,nosuid,noexec,relatime,size=1967912k,nr_inodes=491978,mode=755,inode64)
    devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
    tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=399508k,mode=755,inode64)
    /dev/sda5 on / type ext4 (rw,relatime,errors=remount-ro)
    tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noexec,seclabel)

The option can be temporarily removed `noexec` using mount to 'remount' the directory:

shell

    $ mount -oremount,exec /tmp

To permanently remove the `noexec` flag, update `/etc/fstab` to remove the flag from the options:

shell

    tmpfs /tmp tmpfs mode=1777,nosuid,nodev 0 0

#### Enter search term