Installing On Bare Metal or Cloud Instance

LogScale can be installed and deployed in self-hosted environments using either bare metal (directly onto physical machines) or onto cloud instances (such as Amazon ECS). Installation on these environments requires additional steps and configuration to prepare the systems, install the different components and then install LogScale.

LogScale is deployed as a cluster of machines, typically with a minimum of three nodes. These nodes operate together to provide the LogScale functionality. Higher ingest and retention volumes require larger clusters. A load balancer is placed in front of the cluster to provide a single URL endpoint for communicating with the cluster. A Kafka cluster is required by LogScale to support the ingestion and inter-node communication.

graph LR C[Client] subgraph LB [Load Balancer] direction LR P1[Proxy Host] P2[Proxy Host] P3[Proxy Host] end subgraph LC [LogScale Cluster] direction LR H1[LogScale Node] S1[Local Storage] H2[LogScale Node] S2[Local Storage] H3[LogScale Node] S3[Local Storage] end H1<-->S1 H2<-->S2 H3<-->S3 subgraph KC [Kafka Cluster] direction LR K1[Kafka Node] K2[Kafka Node] K3[Kafka Node] end BS([Block Storage]) C-->LB LB-->LC LC-->KC LC-->BS

The roles and requirements for a successful LogScale deployment:

  • Kafka Cluster to support the ingest and communication between LogScale nodes. See Kafka Usage for more information.

  • LogScale nodes.

  • Load Balancer to provide a single access point and distribution of work across the LogScale cluster.

For background information on LogScale and the overall architecture, see LogScale Internal Architecture.

The full set of processes that needs to be followed is outlined below:

  • System Requirements

    Identify the system requirements for the nodes where Kafka and LogScale will be installed.

  • Preparing for Installation

    Prepare the hosts for installation by configuring the operating system, users and storage.

  • Individual Node Configuration

    Each node in the cluster, in addition to the requirements and preparation sections, will need some additional considerations to ensure that the running systems are available and accessible on the network by ensuring firewalls and instances are configured correctly.

  • Installing Java

    LogScale is written in Scala and required a Java Virtual Machine to execute.

  • Deploy an Apache Kafka Cluster

    LogScale uses Apache Kafka internally for queuing incoming messages, and for storing shared state between nodes. Depending on the Kafka version and variety being used, ZooKeeper may be also be required.

    Depending on the environment, two choices are described here:

  • Installing LogScale

    The LogScale software itself will need installing and configuration.