Falcon LogScale 1.63.5 LTS (2023-03-06)

Version?Type?Release Date?Availability?End of Support

Security

Updates

Upgrades

From?

Config.

Changes?
1.63.5LTS2023-03-06

Cloud

2023-11-30No1.30.0No

Hide file hashes

Show file hashes

Download: https://repo.humio.com/repository/maven-releases/com/humio/server/1.63.5/server-1.63.5.tar.gz

These notes include entries from the following previous releases: 1.63.1, 1.63.2, 1.63.3, 1.63.4

Security fix.

Removed

Items that have been removed as of this release.

Installation and Deployment

  • Deprecated feature removal: the file-based backup feature was deprecated in 1.42.0 and is now removed from Humio. The following configs are no longer supported and will do nothing if set:

    The DELETE_BACKUP_AFTER_MILLIS config is still supported, as it is used for configuring the delay between a file being marked for deletion in Humio, and that file being removed from bucket storage.

Upgrades

Changes that may occur or be required during an upgrade.

  • Other

    • Kafka client has been upgraded to 3.4.0.

      Kafka broker has been upgraded to 3.4.0 in the Kafka container.

      The container upgrade is performed for security reasons to resolve CVE-2022-36944 issue, which Kafka should however not be affected by. If you wish to do a rolling upgrade of your Kafka containers, please always refer to Kafka upgrade guide.

New features and improvements

  • Security

    • The version of Jackson has been upgraded to address CVE-2022-42003 vulnerability.

  • Falcon Data Replicator

  • UI Changes

    • Humio is now a Falcon product. The Humio owl logo and icons are therefore replaced by beautiful falcons.

    • Change Humio logo to Falcon LogScale on login and signup pages.

    • Interactions on JSON data now enabled for JSON arrays in the Event List.

    • Parsing JSON arrays in drill-down context menus no longer adds a trailing dot to the prefix field name.

    • The Single Value widget has updated properties:

      • New design for the toggle switch: it is now bigger and has a green/gray color profile instead of blue/gray.

      • The color profile of the displayed value by trend is now customizable.

    • Following its name change, mentions of Humio have been changed to Falcon LogScale.

    • Add Falcon LogScale announcement on login and signup pages.

    • Contextual drill-down menus for field interactions have been introduced, see Field Interactions. In particular:

      • Fields in the Inspection Panel are now provided with Drill down and Copy context menu items, replacing the former + - GroupBy buttons, see updates at Inspecting Events.

      • The Fields Panel on the left-hand side of the User Interface is now provided with Drill down and Copy context menu items, replacing the former drill-down buttons in the field details flyout (when clicking a field in the fields menu). See updates at Displaying Fields.

      • Fields that have JSON, URL and Timestamps content will have a Parse drill-down option which will parse the field as a LogScale field.

        Parsing JSON will automatically use the field name as prefix for the new field name.

      • Fields containing numbers (currently JSON only) will have Sum, Max, Min, Max values, and Percentiles drill-down options.

  • Automation and Alerts

    • Added two new message templates to actions, {query_start_s} and {query_end_s}. See Message Templates and Variables for details.

    • Self-hosted only: the old implementation of how alert queries are run has been removed. As a consequence, the dynamic configuration UseLegacyAlertJob has also been removed.

  • GraphQL API

  • Dashboards and Widgets

    • JSON in Log Line and JSON formats columns in Event List widgets now have fields underlined on hover and are clickable. This allows drill-downs and copying values easily.

  • Functions

    • QueryAPI — Added staticMetaData property to QueryJobStartedResult. At the moment it only contains the property executionMode, which can be used to communicate hints about the way the backend executes the query to the front-end.

    • Improved the format():

      • Fixed an issue where the format() function would output the wrong amount of left padded zeros for decimal conversions.

      • Formatting large positive numbers as hex no longer causes a loss of bits for integers less than 2^63.

      • Formatting negative numbers as hex no longer produces unintelligible strings.

      • Fixed an issue where adding the # flag would not display the correct formatting string.

      • Fixed an issue where specifying the time/date modifier N would fail to parse.

      • Fixed an issue where supplying multiple fields required you to specify the index of the last field as an argument specifier.

      • Added a length specifier to allow for outputting fields as 32-bit integers instead of 64-bits.

      • Using the type specifier %F now tries to format the specified field as a floating point.

      See the format() reference documentation page for all the above mentioned updates on the supported formatting syntax.

    • QueryAPI — executionModeHint renamed to executionMode.

    • Introduced new valid array syntax in array:contains() and array:regex() functions:

      • Changed the expected format of the array parameter.

      • Changed these functions to no longer be experimental.

  • Other

    • Add code to ensure all mini-segments for the same target end up located on the same hosts. A change in 1.63 could create a situation where mini-segments for the same merge target wound up on different nodes, which the query code currently assumes can't happen. This could cause Result is partial responses to user queries.

    • When selecting a parser test case, the selected test case is highlighted in the UI, so you can see what is selected.

    • Added a new dynamic configuration UndersizedMergingRetentionPercentage, with a default value of 20. This configuration value is used when selecting undersized segments to merge, this setting controls how wide a time span can be merged together.

      The setting is interpreted as a percentage of the repository's retention by time setting. A reasonable range is 0 through to 90.

    • New background task that runs at startup. It verifies the checksums present in local segment files, traversing the most recently updated segment files on the local disk, using the timestamps they have when Humio status. If a file has invalid checksum it will be renamed to crc-error.X where X is the ID of the segment. An error will be logged as well.

    • Add an additional validation check when uploading files to S3-like bucket storage. Humio will now perform a HEAD request for the file's final location in the bucket to verify that the upload succeeded.

    • Added use of the HTTP Proxy Client Configuration, if configured, in a lot of places.

    • Add a script in the tarball distribution's bin directory to check the execution environment, checking common permission issues and other requirements for an environment suitable for running LogScale.

    • Added a new ingest endpoint for receiving metrics and traces via OpenTelemetry OTLP/http. See Ingesting with OpenTelemetry for all the details.

    • Empty datasource directories will be now removed from the local file system while starting the server.

    • Created new test function for event forwarders, which takes as input an event forwarder configuration and tests whether it is possible to connect to the Kafka server. The current test function which takes an ID as input and tests an existing event forwarder by ID, is now marked as deprecated.

    • Use latest version of Java 17 in Docker images.

    • It is now possible to expand multiple bell notifications.

Fixed in this release

  • Security

    • Update Netty to address CVE-2022-41915.

  • UI Changes

    • URL paths with repository name and no trailing /search resolved to Not Found. The URL /repoName will now again show the search page for the repoName repository.

    • Change missing @timestamp field to give a warning instead of an error in functions tail(), head(), bucket(), and timeChart().

  • Automation and Alerts

    • Fixed a bug where a link in the notification for a failed alert would link to a non-existing page.

  • Configuration

  • Dashboards and Widgets

    • Fixed a bug where query result containing no valid results was handled incorrectly in visualisation.

    • Bug fixed in Scatter Chart widget tooltip, so that the description of the actual point only is shown in the tooltip when hovering the mouse over one point, instead of multiple points.

  • Functions

    • Fixed an issue where match() would sometimes give errors when ignoreCase=true and events contained latin1 encoded characters.

    • Fixed an issue where NaN values could cause groupBy() queries to fail.

    • Fixed a bug where the selfJoin() function would not apply the postfilter parameter.

  • Other

    • Unlimited waits for nodes to get in sync has been fixed. This caused digest coordination to fail, to limit the time allowed for a node to get "in sync" on a partition before leadership was assigned to it, in cases where the previous digest leader shut down gracefully.

    • Fixed an issue where nothing was displayed on the average ingest chart in case only one datapoint is present.

    • It is now possible for a user to use the same personal invite token after the user has been transferred to another organization.

    • When selecting a parser test case, the selected test case is highlighted in the UI, so you can see what is selected.

    • Fixed a bug in decryption code used when decrypting downloaded files from bucket storage when version-for-bucket-writes=3. The bug did not allow to decrypt files larger than 2GB.

    • Fixed a regression causing a reflective method lookup to fail when Humio is running on a Java prior to 13.

    • When a host is removed from global, a job tries to clean up any references to it from other places in global, such as segments. Fixed a bug in this job that meant it didn't clean up references on segments that were tombstoned but not yet gone from global. This could block cleanup of those segments.

    • Fix an issue that could cause event redaction tasks to fail to complete, if a segment having events redacted was deleted due to retention.

    • Fix an issue causing a content-length check for bucket uploads to fail when encryption was enabled. The content-length check is not normally enabled, so this should only affect clusters that have disabled ETag-based validation.

    • Fixed an issue where LogScale could log secrets to the debug log when configured to use LDAP or when configured to use SSL for Kafka.

Known Issues