Falcon LogScale 1.118.3 LTS (2024-02-06)
Version? | Type? | Release Date? | Availability? | End of Support | Security Updates | Upgrades From? | Config. Changes? |
---|---|---|---|---|---|---|---|
1.118.3 | LTS | 2024-02-06 | Cloud | 2025-01-31 | No | 1.70.0 | No |
Hide file hashes
TAR Checksum | Value |
---|---|
MD5 | bd20b91e48cfe6dddc10b7554d6a3ad1 |
SHA1 | bba9db6f0ed1dbe63991004479e140fd88162151 |
SHA256 | 05868147bf56827b465b5acb8a264e8e0ec3db29c58449002b01f2a7cdc483c4 |
SHA512 | 53961c1fafa99f0075915582d221c41f8281d50050b5bb11193a4e315053c9bc0267d3dd040837245b2309f481a17807a9bacca03ce9ad3e8f5368c7be21191a |
Download: https://repo.humio.com/repository/maven-releases/com/humio/server/1.118.3/server-1.118.3.tar.gz
These notes include entries from the following previous releases: 1.118.2
Bug fixes and updates.
Breaking Changes
The following items create a breaking change in the behavior, response or operation of this release.
Functions
The new parameter
unit
is added toformatTime()
to specify whether the input field is in seconds or milliseconds, or if it should be auto-detected by the system.This is a breaking change: if you want to ensure fully backward-compatible behavior, set
unit=milliseconds
.For more information, see
formatTime()
.
Advance Warning
The following items are due to change in a future release.
Installation and Deployment
We intend to drop support for Java 17, making Java 21 the minimum. We plan to make this change in March 2024.
Removed
Items that have been removed as of this release.
API
The deprecated REST endpoints
api/v1/dataspaces/(id)/deleteevents
and/api/v1/repositories/(id)/deleteevents
have been removed. You can use the redactEvents GraphQL mutation and query instead.For more information, see redactEvents() .
Deprecation
Items that have been deprecated and may be removed in a future release.
GraphQL mutation updateOrganizationMutability is deprecated in favor of the new setBlockIngest mutation.
Behavior Changes
Scripts or environment which make use of these tools should be checked and updated for the new configuration:
Automation and Alerts
We have changed how Scheduled Searches handle query warnings, similar to what was done for Standard Alerts (see Falcon LogScale 1.112.0 GA (2023-10-24)). Previously, LogScale only triggered Scheduled Searches if there were no query warnings. Now, scheduled searches will trigger despite most query warnings, and the scheduled search status will show a warning instead of an error.
For query warnings about missing data, either due to ingest delay or some existing data that is currently unavailable, the scheduled search will retry for up to 10 minutes by default. This waiting time is configurable, see
SCHEDULED_SEARCH_MAX_WAIT_FOR_MISSING_DATA
for more information.Up until now, all query warnings were treated as errors: the scheduled search did not trigger even though it produced results, and the scheduled search was shown with an error in LogScale. Most query warnings meant that not all data was queried. The previous behaviour prevented the scheduled search from triggering in cases where it would not have, if all data had been available. For instance, a scheduled search that would trigger if a count of events dropped below a threshold. On the other hand, it made some scheduled searches not trigger, even though they would still have if all data was available. That meant that previously you would almost never have a scheduled search trigger when it should not, but you would sometimes have a scheduled search not trigger, when it should have. We have reverted this behavior.
With this change, we no longer recommend to set the configuration option
SCHEDULED_SEARCH_DESPITE_WARNINGS
totrue
, since it treats all query warnings as non-errors, and there are a few query warnings that should make the scheduled search fail.
Upgrades
Changes that may occur or be required during an upgrade.
Configuration
We've migrated from Akka dependency component to Apache Pekko. This means that all internal logs referencing Akka will be substituted with the Pekko counterpart. Users will need to update any triggers or dashboards that rely on such logs.
On Prem only: be aware that the Akka to Pekko migration also affects configuration field names in
application.conf
. Clusters that are using a customapplication.conf
will need to update their configuration to use the Pekko configuration names instead of the Akka configuration names.
New features and improvements
UI Changes
The
Files
page has a new layout and changes:It has been split into two pages: one containing a list of files and one with details of each file.
A view limit of 100 MB has been added and you'll get an error in the UI if you try to view files larger than this size.
It displays information on the size limits and the step needed for syncing the imported files.
For more information, see Files.
Parser test cases will automatically expand to the height of their content when loading the parser page now.
When selecting a parser test case, there is now a button to scroll to that test case again if you scroll away from it.
We have improved the navigation on the page for Alerts, Scheduled Searches and Actions and the page is now called
Automation
.For more information, see Automation.
Lookup Files require unique column headers to work as expected, which was previously validated when attempting to use the file. You could still install an invalid file into LogScale however, but now lookup files with duplicate header names are also blocked from being installed.
Automation and Alerts
LogScale now creates notifications for alerts and scheduled searches with warnings in addition to notifications for errors. The notifications for warnings will have a severity of warning.
When Filter Alerts encounter a query warning that could potentially affect the result of the alert, the warning is now saved with the alert, so that it is visible in the alerts overview, same as for Standard Alerts.
When clearing errors on alerts or scheduled searches, all notifications about the problem are now automatically deleted right when the error is cleared. Previously, notifications were only updated every 15 minutes. Note, that if the error returns, a new notification will be created.
GraphQL API
The redactEvents() mutation will no longer be allowed for users who have a limiting query prefix.
Added limits for GraphQL queries on the total number of selected fields and fragments. Defaults are
1000
for authenticated and150
for unauthenticated users.Cluster administrators can adjust these limits with the
GraphQLSelectionSizeLimit
andUnauthenticatedGraphQLSelectionSizeLimit
dynamic configurations.The new setBlockIngest GraphQL mutation is introduced to block ingest for the organization and set ingest to
paused
in the dataspaces owned by the organization.
Storage
Handling of IOExceptions in part of the segment reading code has been improved. Such exceptions will cause the segment to be excluded from the query, and potentially refetched from bucket storage, and a warning to be shown to the user, rather than cancelling the query.
Configuration
Added validation for
LOCAL_STORAGE_PERCENTAGE
configuration against thetargetDiskUsagePercentage
, that might be set on runtime, to enforce that theLOCAL_STORAGE_PERCENTAGE
variable is at least 5 percentage points larger thantargetDiskUsagePercentage
. Nodes that are violating this constraint will not be able to start. In addition, the setTargetDiskUsagePercentage mutation will not allow violating the constraint.QueryMemoryLimit
andLiveQueryMemoryLimit
dynamic configurations have been replaced withQueryCoordinatorMemoryLimit
, which controls the maximum memory usage of the coordinating node. This memory limit will, in turn, determine the limits of the static query state size and the live query state size.QueryCoordinatorMemoryLimit
defaults to 400MB;QueryMemoryLimit
andLiveQueryMemoryLimit
defaults to 100MB regardless of their previous configuration.For more information, see General Limits & Parameters.
The new
INITIAL_DISABLED_NODE_TASK
environment variable is introduced.For more information, see
INITIAL_DISABLED_NODE_TASK
.
Dashboards and Widgets
Small multiples functionality is introduced for the
Single Value
,Gauge
, andPie Chart
widgets. This feature allows you to partition your query result on a single dimension into multiple visuals of the same widget type for easy comparison.For more information, see Widgets.
We have added the new width option Fit to content for
Event List
columns. With this option selected, the width of the column depends on the content in the column.Table
widget.
Ingestion
When navigating between parser test cases, the table showing the outputs for the test case will now scroll to the top when you select a new test case.
A new mechanism is introduced that delays the response to a HTTP ingest request from nodes that also do digest when the digest node locally experiences digest lag. The following new dynamic configurations control this mechanism:
DelayIngestResponseDueToIngestLagMaxFactor
limits how much longer than the actual execution it may be, measured as a factor on top of the actual time spent (default is2
).DelayIngestResponseDueToIngestLagThreshold
sets the number of milliseconds of digest lag where the feature starts to kick in (default is20,000
).DelayIngestResponseDueToIngestLagScale
sets the number of milliseconds of lag that adds1
to the factor applied (default is300,000
).
The amount of logging produced by
DigestLeadershipLoggerJob
has been reduced in clusters with many ingest queue partitions.
Functions
The new query function
duration()
is introduced: it can be helpful in computations involving timestamps.Live queries that use files in either
match()
,cidr()
, orlookup()
functions are no longer restarted when the file is updated. Instead the files are swapped while the queries are still running.For more information, see Lookup Files Operations.
The new query function
parseUri()
is introduced to support parsing of URIs without a scheme.The new query function
if()
is introduced to compute one of two expressions depending on the outcome of a test.
Fixed in this release
UI Changes
Turned the dropdown menu in the TablePage upwards and set it to the front to fix a bug where the menu would be hidden.
The page for creating repository or view tokens would fail to load if the user didn't have a
Change IP filters
Organization settings permission.
Automation and Alerts
If a filter alert, standard alert or scheduled search was assigned to run on another node in the cluster, due to changes to the available cluster nodes, they would be wrongly marked as failing with an error like The alert is broken. Save the alert again to fix it and an error log. This issue is now fixed.
If an error occurred where the error message was huge, the error would not be stored on the failing alert or scheduled search. This issue has been fixed.
GraphQL API
Swapped parameters in GraphQL mutation updateOrganizationMutability have been fixed.
Storage
A case where we might ignore
LOCAL_STORAGE_PREFILL_PERCENTAGE
and prefetch bucketed segments even if above the limit has been fixed.Fixed an issue that could cause repositories undeleted using the mechanism described at Restoring a Repository or View to be only partially restored. Some deleted datasources within the repositories could erroneously be skipped during restoration.
For more information, see Restoring a Repository or View.
The setTargetDiskUsagePercentage mutation has been removed. The functionality that used this value has been updated to instead base decision-making on
PRIMARY_STORAGE_MAX_FILL_PERCENTAGE
, and onSECONDARY_STORAGE_MAX_FILL_PERCENTAGE
for nodes with secondary storage configured.
Dashboards and Widgets
The
Gauge
widget has been fixed as the Styling panel would not display configured thresholds.Users were prevented from exporting results of queries containing multi value parameters. This issue is now fixed.
The options for precision and thousands separators in
Table
widget have been fixed as they would not be saved correctly when editing other widgets on theSearch
page.The hovered series in
TimeChart
widget have been fixed as they would not be highlighted in the tooltip.The legend title in widget charts has been fixed as it would offset the content when positioned to the right.
The Styling panel in the
Table
widget has been fixed as threshold coloring could be assigned unintentionally.
Ingestion
Parser timeout errors on ingested events that would occur at shutdown have now been fixed.
A gap in the statistics of ingest per day experienced by some organizations on the Usage Page and in humio-usage repository, causing the graph to drop to zero, has now been fixed. As a consequence of this fix, the first measurement performed with version 1.114 will result in the graph showing a peak, since it would include statistics from the period where calculations were skipped.
A parser that failed to construct would sometimes result in events receiving a null error. This issue has been fixed.
A digest coordination issue has been fixed: it could cause mini-segments to stay behind on old digest leaders when leadership changes.
Queries
Occasional error logging from
QueryScheduler.reduceAndSetSnapshot
has been fixed.
Functions
cidr()
query function would fail to find some events when parameternegate=true
was set. This incorrect behavior has now been fixed.The
cidr()
function would handle a validation error incorrectly. This issue has been fixed.The
count()
function withdistinct
parameter would give an incorrect count forutf8
strings. This issue has been fixed.timeChart()
andbucket()
functions have been fixed as they would give slightly different results depending on whether theirlimit
argument was left out or explicitly set to the default value.