Skip to content
LogScale Self-Hosted Deployment
This is an archived version of the documentation downloaded from LogScale Documentation Archives. Up to date documentation, and documentation for later versions, is available on LogScale Documentation

## Switching Kafka

LogScale uses Kafka for queuing incoming messages and for storing shared state when running LogScale in a cluster setup. It's possible for LogScale to snapshot its state and continue running using a new Kafka cluster. This can be useful in situations where you want to change infrastructure or if there are problems with the current Kafka cluster. This section describes the procedure for doing a Kafka switch.

### Danger

All LogScale processes must be completely stopped before performing this action.

### Stop Sending Data to LogScale

If it is possible, stop sending data to LogScale, then wait for LogScale to process all data on the ingest queue. The LogScale Stats dashboard in the LogScale repository have an Events Processed After Ingest Queue graph by host per second and an Ingest Latency graph. If there is data on the ingest queue after closing LogScale it will be lost as the queue is reset or another queue will be used.

You'll need to stop all LogScale processes on all machines. You'll also need to stop all Kafka processes and all ZooKeeper processes on all machines.

The next step depends on the deployment model:

- When using Kafka deployed with ZooKeeper (up to LogScale 1.107)

  - Switch Kafka and ZooKeeper

  - Restarting Kafka, ZooKeeper and LogScale

- When using Kafka deployed with KRaft

  - Switch Kafka using KRaft Mode

  - Restarting Kafka using KRaft and LogScale

### Switching Kafka Recap

While detailed instructions are provided for switching a Zookeeper-based cluster or switching a Kafka KRaft cluster, the following provides a simplified overview of the steps required to switch a Kafka cluster:

1. Set `ALLOW_KAFKA_RESET_UNTIL_TIMESTAMP_MS` to enable Kafka resets (it is unset by default).

2. Stop all LogScale processes on all nodes.

3. Stop all Kafka processes on all nodes.

4. Stop all ZooKeeper processes on all nodes (up to LogScale 1.107).

5. Delete Kafka data (or use new Kafka queues). If using KRaft, make sure you set the Kafka cluster ID correctly when resetting the cluster, as described in the Kafka documentation.

6. Delete ZooKeeper data ((up to LogScale 1.107)).

7. Start all ZooKeeper processes on all nodes (up to LogScale 1.107).

8. Verify the ZooKeeper cluster (up to LogScale 1.107).

9. Start all Kafka processes on all nodes.

10. Verify the Kafka cluster.

11. Start one LogScale node and let it change epoch.

12. Verify the epoch has changed.

13. Start the other LogScale processes on all nodes.

#### Enter search term