Falcon LogScale 1.143.0 Preview (2024-06-18)

Version?Type?Release Date?Availability?End of Support

Security

Updates

Upgrades

From?

JDK

Compatibility?

Config.

Changes?
1.143.0Preview2024-06-18

Cloud

Next StableNo1.11221-22No

Bug fixes and updates.

Advance Warning

The following items are due to change in a future release.

  • Installation and Deployment

    • The LogScale Launcher Script script for starting LogScale will be modified to change the way CPU core usage can be configured. The -XX:ActiveProcessorCount=n command-line option will be ignored if set. Users that need to configure the core count manually should set CORES=n environment variable instead. This will cause the launcher to configure both LogScale and the JVM properly.

      This change is scheduled for 1.148.0.

      For more information, see Configuring Available CPU Cores.

Removed

Items that have been removed as of this release.

Other

  • Unnecessary digest-coordinator-changes and desired-digest-coordinator-changes metrics have been removed. Instead, the logging in the IngestPartitionCoordinator class has been improved, to allow monitoring of when reassignment of desired and current digesters happens — by searching for Wrote changes to desired digest partitions / Wrote changes to current digest partitions.

Deprecation

Items that have been deprecated and may be removed in a future release.

  • The following API endpoints are deprecated and marked for removal in 1.148.0:

    • POST /api/v1/clusterconfig/kafka-queues/partition-assignment

    • GET /api/v1/clusterconfig/kafka-queues/partition-assignment

    • POST /api/v1/clusterconfig/kafka-queues/partition-assignment/set-replication-defaults

    The deprecated methods are used for viewing and changing the partition assignment in Kafka for the ingest queue. Administrators should use Kafka's own tools for editing partition assignments instead, such as the bin/kafka-reassign-partitions.sh and bin/kafka-topics.sh scripts that ship with the Kafka install.

  • The server.tar.gz release artifact has been deprecated. Users should switch to the OS/architecture-specific server-linux_x64.tar.gz or server-alpine_x64.tar.gz, which include bundled JDKs. Users installing a Docker image do not need to make any changes. With this change, LogScale will no longer support bringing your own JDK, we will bundle one with releases instead.

    We are making this change for the following reasons:

    • By bundling a JDK specifically for LogScale, we can customize the JDK to contain only the functionality needed by LogScale. This is a benefit from a security perspective, and also reduces the size of release artifacts.

    • Bundling the JDK ensures that the JDK version in use is one we've tested with, which makes it more likely a customer install will perform similar to our own internal setups.

    • By bundling the JDK, we will only need to support one JDK version. This means we can take advantage of enhanced JDK features sooner, such as specific performance improvements, which benefits everyone.

    The last release where server.tar.gz artifact is included will be 1.154.0.

  • We are deprecating the humio/kafka and humio/zookeeper Docker images due to low use. The planned final release for these images will be with LogScale 1.148.0.

    Better alternatives are available going forward. We recommend the following:

    • If your cluster is deployed on Kubernetes: STRIMZI

    • If your cluster is deployed to AWS: MSK

    If you still require humio/kafka or humio/zookeeper for needs that cannot be covered by these alternatives, please contact Support and share your concerns.

  • The HUMIO_JVM_ARGS environment variable in the LogScale Launcher Script script will be removed in 1.154.0.

    The variable existed for migration from older deployments where the launcher script was not available. The launcher script replaces the need for manually setting parameters in this variable, so the use of this variable is no longer required. Using the launcher script is now the recommended method of launching LogScale. For more details on the launcher script, see LogScale Launcher Script. Clusters that still set this configuration should migrate to the other variables described at Override garbage collection configuration within the launcher script.

Upgrades

Changes that may occur or be required during an upgrade.

  • Installation and Deployment

    • The minimum version of Java compatible with LogScale is now 21. Docker users, and users installing the release artifacts that bundle the JDK, are not affected.

      It is recommended to switch to the release artifacts that bundle a JDK, because LogScale no longer supports bringing your own JDK as of release 1.138, see Falcon LogScale 1.138.0 Preview (2024-05-14)

New features and improvements

  • Security

    • When extending Retention span or size, any segments that were marked for deletion — but where the files remain in the system — are automatically resurrected. How much data you reclaim via this depends on the backupAfterMillis configuration on the repository.

      For more information, see Audit Logging.

  • GraphQL API

    • The new concatenateQueries() GraphQL API has been introduced for programmatically concatenating multiple queries into one. This is intended to eliminate errors that might occur if queries are combined naively.

    • The new environmentVariableUsage() GraphQL API has been introduced for listing non-secret environment variables used by a node. This is intended as an aid to help do configuration discovery when managing a large number of LogScale clusters.

    • The preview tag has been removed from the following GraphQL mutations:

      • createAwsS3SqsIngestFeed

      • DeleteIngestFeed

      • resetQuota

      • testAwsS3SqsIngestFeed

      • triggerPollIngestFeed

      • updateAwsS3SqsIngestFeed

  • Functions

    • The match() function now supports matching on multiple pairs of fields and columns.

      For more information, see match().

Fixed in this release

  • UI Changes

    • In the Export to File dialog, when using the keyboard to switch between options, a different item than the one selected was highlighted. This issue has now been fixed.

  • Storage

    • Digest threads could fail to start digesting if global is very large, and if writing to global is slow. This issue has now been fixed.