Falcon LogScale Documentation

Other

• OpenSSL in Docker images has been upgraded to address CVE-2023-0286 issue.

Other

• Upgrade Kafka client to 3.4.0.

  Upgrade Kafka broker to 3.4.0 in the Kafka container.

  The container upgrade is performed for security reasons to resolve CVE-2022-36944 issue, which Kafka should however not be affected by. If you wish to do a rolling upgrade of your Kafka containers, please always refer to Kafka upgrade guide.

Other

• Upgrade Kafka client to 3.4.0.

  Upgrade Kafka broker to 3.4.0 in the Kafka container.

  The container upgrade is performed for security reasons to resolve CVE-2022-36944 issue, which Kafka should however not be affected by. If you wish to do a rolling upgrade of your Kafka containers, please always refer to Kafka upgrade guide.

Other

• Upgrade Java to 17.0.6 in Docker containers

  Upgrade Kafka to 3.3.2 for KAFKA-14379

  Upgrade Kafka client to 3.3.2

  Upgrade Kafka Docker container to 3.3.2

Packages

• Optimizations in package handling require migration of data during upgrade. This migration is performed automatically. Please notice:

  • While the upgrade of cluster nodes are ongoing, we recommend you do not install or update any packages, as they may end up in an inconsistent state.

    If a package ends up in a bad state during migration, it can be fixed simply by reinstalling the package.

  • You will potentially experience that accessing the list of installed packages will fail, and creating new dashboards, alerts, parsers, etc. based on package templates will not work as intended.

    This should only happen during the cluster upgrade, and should resolve itself once the cluster is fully upgraded.

  • If the cluster nodes are downgraded, any packages installed or updated while running the new version will not work, and we therefore recommend uninstalling or downgrading those packages prior to downgrading the cluster nodes.

Other

• Upgrade Java to 17.0.6 in Docker containers

  Upgrade Kafka to 3.3.2 for KAFKA-14379

  Upgrade Kafka client to 3.3.2

  Upgrade Kafka Docker container to 3.3.2

Packages

• Optimizations in package handling require migration of data during upgrade. This migration is performed automatically. Please notice:

  • While the upgrade of cluster nodes are ongoing, we recommend you do not install or update any packages, as they may end up in an inconsistent state.

    If a package ends up in a bad state during migration, it can be fixed simply by reinstalling the package.

  • You will potentially experience that accessing the list of installed packages will fail, and creating new dashboards, alerts, parsers, etc. based on package templates will not work as intended.

    This should only happen during the cluster upgrade, and should resolve itself once the cluster is fully upgraded.

  • If the cluster nodes are downgraded, any packages installed or updated while running the new version will not work, and we therefore recommend uninstalling or downgrading those packages prior to downgrading the cluster nodes.

Other

• Upgrade Kafka client to 3.4.0.

  Upgrade Kafka broker to 3.4.0 in the Kafka container.

  The container upgrade is performed for security reasons to resolve CVE-2022-36944 issue, which Kafka should however not be affected by. If you wish to do a rolling upgrade of your Kafka containers, please always refer to Kafka upgrade guide.

Installation and Deployment

• We have enabled a new vhost selection method by default. The way hosts select their vhost number when joining the cluster has changed, the new logic is described at Node Identifiers documentation page.

  The new logic does not depend on Zookeeper, even for clusters where nodes occasionally lose disk contents, such as Kubernetes. In order to smooth migration for clusters using Zookeeper, the new logic will still interact with Zookeeper to avoid nodes using a mix of new and old vhost code from fighting over the vhost numbers. This is only necessary while migrating.

  The recommended steps for migrating off of Zookeeper are as follows:

  1. Deploy the new LogScale version to all nodes.

  2. Remove ZOOKEEPER_URL_FOR_NODE_UUID, ZOOKEEPER_URL, ZOOKEEPER_PREFIX_FOR_NODE_UUID, ZOOKEEPER_SESSIONTIMEOUT_FOR_NODE_UUID from the configuration for all nodes.

  3. Reboot

  Once rebooted, LogScale will no longer need Zookeeper directly, except as an indirect dependency of Kafka. Due to this, the 4 Zookeeper-related variables are deprecated as of this release and will be removed in a future version.

  Since vhost numbers now change when a disk is wiped, cluster administrators for clusters using nodes where USING_EPHEMERAL_DISKS is set to true will need to ensure that the storage and digest partitioning tables are up to date as hosts join and leave the cluster. Updating the tables is handled automatically if using the Logscale Kubernetes operator, but for clusters that do not use this operator, cluster administrators should run scripts periodically to keep the storage and digest tables up to date. This is not a new requirement for ephemeral clusters, but we're providing a reminder here since it may be needed more frequently now.

  The cluster GraphQL query can provide updated tables (the suggestedIngestPartitions and suggestedStoragePartitions fields), which can then be applied via the updateIngestPartitionScheme and updateStoragePartitionScheme GraphQL mutations.

  Should you experience any issue in using this feature, you may opt out by setting NEW_VHOST_SELECTION_ENABLED=false. If you do this, please reach out to support with feedback, as we otherwise intend to remove the old vhost selection logic in the coming months.

  Note

  When using Operator and Kubernetes deployments, you must upgrade to 0.17.0 of operator to support migration away from the Zookeeper requirement. See Version 0.17.0.

Other

• Upgrade Kafka client to 3.4.0.

  Upgrade Kafka broker to 3.4.0 in the Kafka container.

  The container upgrade is performed for security reasons to resolve CVE-2022-36944 issue, which Kafka should however not be affected by. If you wish to do a rolling upgrade of your Kafka containers, please always refer to Kafka upgrade guide.