Upgrading Kafka

This section covers the recommended approach for performing a rolling upgrade of a self-hosted Kafka cluster used with LogScale.

This section applies to clusters running in KRaft mode (separate controllers and brokers) as well as other common topologies. Adjust the sequence as appropriate for your environment.

Prerequisites

Run the upgrade in a User Acceptance Testing (UAT) or staging environment first. Validate that LogScale operates correctly against the upgraded Kafka version before proceeding to production.

See also the Apache Kafka Rolling Upgrade Documentation.

Establish a Rollback Plan

Define a per-node rollback procedure before starting. A full-cluster restore plan alone is insufficient — you should be able to roll back a single broker or controller independently if it fails to rejoin the cluster after upgrade.

Disaster Recovery Preparation

Complete the following steps on all nodes before beginning the upgrade:

  1. Take a full backup or snapshot of Kafka data volumes and configuration files

  2. Export your current LogScale configuration

  3. Verify that your restore procedure works — a backup that has not been tested is not a recovery plan

Note

There is no LogScale-specific Kafka backup and recovery guide. Follow standard Apache Kafka DR best practices in conjunction with your existing LogScale backup procedures.

Rolling Upgrade Procedure

A rolling upgrade is the recommended approach for upgrading Kafka in a production environment. Nodes are upgraded one at a time, allowing the cluster to remain available throughout the process.

Some general principles should be applied:

  • Do not skip multiple major versions in a single upgrade pass if it can be avoided

  • Maintain inter-broker protocol version compatibility throughout the transition

  • Confirm each node has fully rejoined the cluster and is healthy before proceeding to the next node

Upgrade Sequence by Topology

For KRaft Mode (Separate Controllers and Brokers):

  1. Upgrade controllers one at a time, confirming cluster health after each

  2. Once all controllers are upgraded, upgrade brokers one at a time, confirming cluster health after each

For Combined Controller/Broker Nodes:

  1. Upgrade nodes one at a time in any consistent order

  2. Confirm each node rejoins the cluster and resumes normal operation before proceeding

Validating Cluster Health After Each Node Upgrade

Before moving to the next node, confirm the following:

  • The upgraded node has rejoined the cluster

  • Partition leadership has rebalanced as expected

  • No under-replicated partitions remain

  • LogScale is successfully producing and consuming messages