Falcon LogScale 1.118.4 LTS (2024-02-23)

Version?Type?Release Date?Availability?End of Support

Security

Updates

Upgrades

From?

Config.

Changes?
1.118.4LTS2024-02-23

Cloud

2025-01-31No1.70.0No

Hide file hashes

Show file hashes

Download: https://repo.humio.com/repository/maven-releases/com/humio/server/1.118.4/server-1.118.4.tar.gz

These notes include entries from the following previous releases: 1.118.2, 1.118.3

Bug fixes and performance improvements.

Breaking Changes

The following items create a breaking change in the behavior, response or operation of this release.

  • Functions

    • The new parameter unit is added to formatTime() to specify whether the input field is in seconds or milliseconds, or if it should be auto-detected by the system.

      This is a breaking change: if you want to ensure fully backward-compatible behavior, set unit=milliseconds.

      For more information, see formatTime().

Advance Warning

The following items are due to change in a future release.

  • Installation and Deployment

    • We intend to drop support for Java 17, making Java 21 the minimum. We plan to make this change in March 2024.

Removed

Items that have been removed as of this release.

API

  • The deprecated REST endpoints api/v1/dataspaces/(id)/deleteevents and /api/v1/repositories/(id)/deleteevents have been removed. You can use the redactEvents GraphQL mutation and query instead.

    For more information, see redactEvents() .

Deprecation

Items that have been deprecated and may be removed in a future release.

  • GraphQL mutation updateOrganizationMutability is deprecated in favor of the new setBlockIngest mutation.

Behavior Changes

Scripts or environment which make use of these tools should be checked and updated for the new configuration:

  • Automation and Alerts

    • We have changed how Scheduled Searches handle query warnings, similar to what was done for Standard Alerts (see Falcon LogScale 1.112.0 GA (2023-10-24)). Previously, LogScale only triggered Scheduled Searches if there were no query warnings. Now, scheduled searches will trigger despite most query warnings, and the scheduled search status will show a warning instead of an error.

      For query warnings about missing data, either due to ingest delay or some existing data that is currently unavailable, the scheduled search will retry for up to 10 minutes by default. This waiting time is configurable, see SCHEDULED_SEARCH_MAX_WAIT_FOR_MISSING_DATA for more information.

      Up until now, all query warnings were treated as errors: the scheduled search did not trigger even though it produced results, and the scheduled search was shown with an error in LogScale. Most query warnings meant that not all data was queried. The previous behaviour prevented the scheduled search from triggering in cases where it would not have, if all data had been available. For instance, a scheduled search that would trigger if a count of events dropped below a threshold. On the other hand, it made some scheduled searches not trigger, even though they would still have if all data was available. That meant that previously you would almost never have a scheduled search trigger when it should not, but you would sometimes have a scheduled search not trigger, when it should have. We have reverted this behavior.

      With this change, we no longer recommend to set the configuration option SCHEDULED_SEARCH_DESPITE_WARNINGS to true, since it treats all query warnings as non-errors, and there are a few query warnings that should make the scheduled search fail.

Upgrades

Changes that may occur or be required during an upgrade.

  • Configuration

    • We've migrated from Akka dependency component to Apache Pekko. This means that all internal logs referencing Akka will be substituted with the Pekko counterpart. Users will need to update any triggers or dashboards that rely on such logs.

      On Prem only: be aware that the Akka to Pekko migration also affects configuration field names in application.conf. Clusters that are using a custom application.conf will need to update their configuration to use the Pekko configuration names instead of the Akka configuration names.

New features and improvements

  • UI Changes

    • The Files page has a new layout and changes:

      • It has been split into two pages: one containing a list of files and one with details of each file.

      • A view limit of 100 MB has been added and you'll get an error in the UI if you try to view files larger than this size.

      • It displays information on the size limits and the step needed for syncing the imported files.

      For more information, see Files.

    • Parser test cases will automatically expand to the height of their content when loading the parser page now.

    • When selecting a parser test case, there is now a button to scroll to that test case again if you scroll away from it.

    • We have improved the navigation on the page for Alerts, Scheduled Searches and Actions and the page is now called Automation.

      For more information, see Automation.

    • Lookup Files require unique column headers to work as expected, which was previously validated when attempting to use the file. You could still install an invalid file into LogScale however, but now lookup files with duplicate header names are also blocked from being installed.

  • Automation and Alerts

    • LogScale now creates notifications for alerts and scheduled searches with warnings in addition to notifications for errors. The notifications for warnings will have a severity of warning.

    • When Filter Alerts encounter a query warning that could potentially affect the result of the alert, the warning is now saved with the alert, so that it is visible in the alerts overview, same as for Standard Alerts.

    • When clearing errors on alerts or scheduled searches, all notifications about the problem are now automatically deleted right when the error is cleared. Previously, notifications were only updated every 15 minutes. Note, that if the error returns, a new notification will be created.

  • Storage

    • Handling of IOExceptions in part of the segment reading code has been improved. Such exceptions will cause the segment to be excluded from the query, and potentially refetched from bucket storage, and a warning to be shown to the user, rather than cancelling the query.

  • GraphQL API

    • The redactEvents() mutation will no longer be allowed for users who have a limiting query prefix.

    • Added limits for GraphQL queries on the total number of selected fields and fragments. Defaults are 1000 for authenticated and 150 for unauthenticated users.

      Cluster administrators can adjust these limits with the GraphQLSelectionSizeLimit and UnauthenticatedGraphQLSelectionSizeLimit dynamic configurations.

    • The new setBlockIngest GraphQL mutation is introduced to block ingest for the organization and set ingest to paused in the dataspaces owned by the organization.

  • Configuration

    • Added validation for LOCAL_STORAGE_PERCENTAGE configuration against the targetDiskUsagePercentage, that might be set on runtime, to enforce that the LOCAL_STORAGE_PERCENTAGE variable is at least 5 percentage points larger than targetDiskUsagePercentage. Nodes that are violating this constraint will not be able to start. In addition, the setTargetDiskUsagePercentage mutation will not allow violating the constraint.

    • QueryMemoryLimit and LiveQueryMemoryLimit dynamic configurations have been replaced with QueryCoordinatorMemoryLimit, which controls the maximum memory usage of the coordinating node. This memory limit will, in turn, determine the limits of the static query state size and the live query state size. QueryCoordinatorMemoryLimit defaults to 400MB; QueryMemoryLimit and LiveQueryMemoryLimit defaults to 100MB regardless of their previous configuration.

      For more information, see General Limits & Parameters.

    • The new INITIAL_DISABLED_NODE_TASK environment variable is introduced.

      For more information, see INITIAL_DISABLED_NODE_TASK.

  • Ingestion

    • When navigating between parser test cases, the table showing the outputs for the test case will now scroll to the top when you select a new test case.

    • A new mechanism is introduced that delays the response to a HTTP ingest request from nodes that also do digest when the digest node locally experiences digest lag. The following new dynamic configurations control this mechanism:

      • DelayIngestResponseDueToIngestLagMaxFactor limits how much longer than the actual execution it may be, measured as a factor on top of the actual time spent (default is 2).

      • DelayIngestResponseDueToIngestLagThreshold sets the number of milliseconds of digest lag where the feature starts to kick in (default is 20,000).

      • DelayIngestResponseDueToIngestLagScale sets the number of milliseconds of lag that adds 1 to the factor applied (default is 300,000).

    • The amount of logging produced by DigestLeadershipLoggerJob has been reduced in clusters with many ingest queue partitions.

  • Dashboards and Widgets

    • Small multiples functionality is introduced for the Single Value, Gauge, and Pie Chart widgets. This feature allows you to partition your query result on a single dimension into multiple visuals of the same widget type for easy comparison.

      For more information, see Widgets.

    • We have added the new width option Fit to content for Event List columns. With this option selected, the width of the column depends on the content in the column.

    • Show thousands separator has been added as a configuration option of format Number for the Table widget.

  • Functions

    • The new query function duration() is introduced: it can be helpful in computations involving timestamps.

    • Live queries that use files in either match(), cidr(), or lookup() functions are no longer restarted when the file is updated. Instead the files are swapped while the queries are still running.

      For more information, see Lookup Files Operations.

    • The new query function parseUri() is introduced to support parsing of URIs without a scheme.

    • The new query function if() is introduced to compute one of two expressions depending on the outcome of a test.

Fixed in this release

  • UI Changes

    • Turned the dropdown menu in the TablePage upwards and set it to the front to fix a bug where the menu would be hidden.

    • The page for creating repository or view tokens would fail to load if the user didn't have a Change IP filters Organization settings permission.

  • Automation and Alerts

    • If a filter alert, standard alert or scheduled search was assigned to run on another node in the cluster, due to changes to the available cluster nodes, they would be wrongly marked as failing with an error like The alert is broken. Save the alert again to fix it and an error log. This issue is now fixed.

    • If an error occurred where the error message was huge, the error would not be stored on the failing alert or scheduled search. This issue has been fixed.

  • Storage

  • GraphQL API

    • Swapped parameters in GraphQL mutation updateOrganizationMutability have been fixed.

  • Ingestion

    • Parser timeout errors on ingested events that would occur at shutdown have now been fixed.

    • A gap in the statistics of ingest per day experienced by some organizations on the Usage Page and in humio-usage repository, causing the graph to drop to zero, has now been fixed. As a consequence of this fix, the first measurement performed with version 1.114 will result in the graph showing a peak, since it would include statistics from the period where calculations were skipped.

    • A parser that failed to construct would sometimes result in events receiving a null error. This issue has been fixed.

    • A digest coordination issue has been fixed: it could cause mini-segments to stay behind on old digest leaders when leadership changes.

  • Dashboards and Widgets

    • The Gauge widget has been fixed as the Styling panel would not display configured thresholds.

    • The hovered series in Time Chart widget have been fixed as they would not be highlighted in the tooltip.

    • Users were prevented from exporting results of queries containing multi value parameters. This issue is now fixed.

    • The options for precision and thousands separators in Table widget have been fixed as they would not be saved correctly when editing other widgets on the Search page.

    • The legend title in widget charts has been fixed as it would offset the content when positioned to the right.

    • The Styling panel in the Table widget has been fixed as threshold coloring could be assigned unintentionally.

  • Queries

    • Occasional error logging from QueryScheduler.reduceAndSetSnapshot has been fixed.

  • Functions

    • cidr() query function would fail to find some events when parameter negate=true was set. This incorrect behavior has now been fixed.

    • The cidr() function would handle a validation error incorrectly. This issue has been fixed.

    • The count() function with distinct parameter would give an incorrect count for utf8 strings. This issue has been fixed.

    • timeChart() and bucket() functions have been fixed as they would give slightly different results depending on whether their limit argument was left out or explicitly set to the default value.

Improvement

  • Storage

    • Allowed reassignment of digest that assigns partitions unevenly to hosts. This is to support clusters where hosts are not evenly sized, and so an even partition assignment is not expected.