Falcon LogScale 1.144.0 Preview (2024-06-25)

Version?Type?Release Date?Availability?End of Support

Security

Updates

Upgrades

From?

JDK

Compatibility?

Config.

Changes?
1.144.0Preview2024-06-25

Cloud

Next StableYes1.11221-22No

Bug fixes and updates.

Advanced Warning

The following items are due to change in a future release.

  • Installation and Deployment

    • The LogScale Launcher Script script for starting LogScale will be modified to change the way CPU core usage can be configured. The -XX:ActiveProcessorCount=n command-line option will be ignored if set. Users that need to configure the core count manually should set CORES=n environmant variable instead. This will cause the launcher to configure both LogScale and the JVM properly.

      This change is scheduled for 1.148.0.

      For more information, see Configuring Available CPU Cores.

Deprecation

Items that have been deprecated and may be removed in a future release.

  • The following API endpoints are deprecated and marked for removal in 1.148.0:

    • POST /api/v1/clusterconfig/kafka-queues/partition-assignment

    • GET /api/v1/clusterconfig/kafka-queues/partition-assignment

    • POST /api/v1/clusterconfig/kafka-queues/partition-assignment/set-replication-defaults

    The deprecated methods are used for viewing and changing the partition assignment in Kafka for the ingest queue. Administrators should use Kafka's own tools for editing partition assignments instead, such as the bin/kafka-reassign-partitions.sh and bin/kafka-topics.sh scripts that ship with the Kafka install.

  • The server.tar.gz release artifact has been deprecated. Users should switch to the OS/architecture-specific server-linux_x64.tar.gz or server-alpine_x64.tar.gz, which include bundled JDKs. Users installing a Docker image do not need to make any changes. With this change, LogScale will no longer support bringing your own JDK, we will bundle one with releases instead.

    We are making this change for the following reasons:

    • By bundling a JDK specifically for LogScale, we can customize the JDK to contain only the functionality needed by LogScale. This is a benefit from a security perspective, and also reduces the size of release artifacts.

    • Bundling the JDK ensures that the JDK version in use is one we've tested with, which makes it more likely a customer install will perform similar to our own internal setups.

    • By bundling the JDK, we will only need to support one JDK version. This means we can take advantage of enhanced JDK features sooner, such as specific performance improvements, which benefits everyone.

    The last release where server.tar.gz artifact is included will be 1.154.0.

  • The HUMIO_JVM_ARGS environment variable in the LogScale Launcher Script script will be removed in 1.154.0.

    The variable existed for migration from older deployments where the launcher script was not available. The launcher script replaces the need for manually setting parameters in this variable, so the use of this variable is no longer required. Using the launcher script is now the recommended method of launching LogScale. For more details on the launcher script, see LogScale Launcher Script. Clusters that still set this configuration should migrate to the other variables described at Override garbage collection configuration within the launcher script.

  • We are deprecating the humio/kafka and humio/zookeeper Docker images due to low use. The planned final release for these images will be with LogScale 1.148.0.

    Better alternatives are available going forward. We recommend the following:

    • If your cluster is deployed on Kubernetes: STRIMZI

    • If your cluster is deployed to AWS: MSK

    If you still require humio/kafka or humio/zookeeper for needs that cannot be covered by these alternatives, please contact Support and share your concerns.

  • The lastScheduledSearch field from the ScheduledSearch datatype is now deprecated and planned for removal in LogScale version 1.202. The new lastExecuted and lastTriggered fields have been added to the ScheduledSearch datatype to replace lastScheduledSearch.

Behavior Changes

Scripts or environment which make use of these tools should be checked and updated for the new configuration:

  • Installation and Deployment

    • The default cleanup.policy for the transientChatter-events topic has been switched from compact to delete,compact. This change will not apply to existing clusters. Changing this setting to delete,compact via Kafka's command line tools is particularly recommended if transientChatter is taking up excessive space on disk, whereas it is less relevant in production environments where Kafka's disks tend to be large.

  • Configuration

    • When global publish to Kafka times out from digester threads, the system would initiate a failure shutdown. Instead, from this 1.144 version the system retries the publish to Global Database indefinitely for those specific global transactions that originate in a digester thread. If retries occur, these get logged with an error executeTransactionRetryingOnTimeout: unable to execute transaction for global, retrying.

New features and improvements

  • Automation and Alerts

    • Two new GraphQL fields have been added in the ScheduledSearch datatype:

      • lastExecuted will hold the timestamp of the end of the search interval on the last scheduled search run.

      • lastTriggered will hold the timestamp of the end of the search interval on the last scheduled search run that found results and triggered actions.

      These two new fields are now displayed in the Scheduled Searches user interface.

  • Storage

    • Support for bucket storage upload validation has changed. LogScale now supports the following three validation modes:

      • Checking the ETag HTTP response header on the upload response. This mode is the default, and can be opted out of via the BUCKET_STORAGE_IGNORE_ETAG_UPLOAD configuration parameter.

      • Checking the ETag HTTP response header on a HEAD request done for the uploaded file. This is the second preferred mode, and can be opted out of via the BUCKET_STORAGE_IGNORE_ETAG_AFTER_UPLOAD configuration parameter.

      • Downloading the file that was uploaded, in order to validate the checksum file. This mode is enabled if neither of the other modes are enabled.

      Previous validation modes that did not compare checksums have been removed, as they were not reliable indicators of the uploaded file integrity.

  • GraphQL API

    • The log line containing Executed GraphQL query in the humio repository, that is logged for every GraphQL call, now contains the name of the mutations and queries that are executed.

Fixed in this release