Falcon LogScale 1.159.1 LTS (2024-10-31)
Version? | Type? | Release Date? | Availability? | End of Support | Security Updates | Upgrades From? | Config. Changes? |
---|---|---|---|---|---|---|---|
1.159.1 | LTS | 2024-10-31 | Cloud On-Prem | 2025-10-31 | Yes | 1.112 | No |
Filename | Hashtype | File Hash |
---|---|---|
server-alpine_x64 | SHA256 | c39192a6e78307694965fda7b36f0be4044b5385a7edaff84e539599a7cd8e70 |
server-linux_x64 | SHA256 | 1a472c88cfdd1bff9b82c6adb495bdde7eb1530274eaed0cedf73640b7892b33 |
Docker Image | SHA256 Checksum |
---|---|
humio-core | d57a3645e5870838097a0128d4e5d2f57d747e544926df486d28cbd6d9ea41f0 |
humio-single-node-demo | 36a40a62e626ba51e52281f871439532ef764fa4a010d8b1f5768c071357e697 |
Download
Bug fixes and updates.
Deprecation
Items that have been deprecated and may be removed in a future release.
The following GraphQL mutations and field have been deprecated, since the starring functionality is no longer in use for alerts and scheduled searches:
addStarToAlertV2
removeStarFromAlertV2
addStarToScheduledSearch
removeStarFromScheduledSearch
isStarred field on the
Alert
andScheduledSearch
types.The lastScheduledSearch field from the
ScheduledSearch
datatype is now deprecated and planned for removal in LogScale version 1.202. The new lastExecuted and lastTriggered fields have been added to theScheduledSearch
datatype to replace lastScheduledSearch.The deprecated JDK-less
server.tar.gz
tarball release is no longer being published. Users should switch to eitherserver-linux_x64.tar.gz
orserver-alpine_x64.tar.gz
depending on their operating system.
Behavior Changes
Scripts or environment which make use of these tools should be checked and updated for the new configuration:
Automation and Alerts
Aggregate Alerts and Filter Alerts as well as Scheduled Searches will now stop the query, if it has become outdated before it finishes.
Storage
LogScale now avoids moving mini-segments to follow the digest nodes if the mini-segments are available in Bucket Storage. Instead, mini-segments will now be fetched as needed, when the digest leader is ready to merge them. This reduces the load on Global Database in some cases following a digest reassignment.
During digest reassignment, LogScale will now ignore mini-segments in Bucket Storage when deciding whether to switch merge targets because some mini-segments are not present locally. This should slightly reduce the load on Global Database during digest reassignment.
Allow live query updates to run on a new thread pool
digestLive
, but only for datasources that spend more time on these updates than allowed in the digester pool on live queries, or for many datasources, if their total load exceeds time available for digesters. This frees up time for the digesters, provided there is available CPU on the node.LogScale now avoids moving merge targets to the digest leader during digest reassignment if those segments are already in Bucket Storage.
Ingestion
Falcon LogScale now improves decision-making around which segments a digest leader fetches as part of taking over leadership. This should reduce the incidence of small bits of data being replayed from Kafka unncessarily, and may also reduce how often reassignment will trigger a restart of live queries.
For more information, see Ingestion: Digest Phase.
Queries
When a digest node is unavailable, a warning is not attached to queries, but the queries are allowed to proceed.
This way, the behaviour of a query is similar to the case where a segment cannot be searched, due to all the owning nodes being unavailable at the time of the query.
Upgrades
Changes that may occur or be required during an upgrade.
Upgrades
Bundled JDK is now upgraded to Java 23.
Installation and Deployment
The JDK has been upgraded to 23.0.1
Upgraded the Kafka clients to 3.8.0.
New features and improvements
Security
New view permissions have been added to allow for updating and deleting different types of assets in a view. For instance, granting a user the
UpdateFiles
permission in a view will allow the user to update files, but not delete or create files.View permissions added:
UpdateActions
– Allow updating actionsUpdateDashboards
– Allow updating dashboardsUpdateFiles
– Allow updating CSV filesUpdateSavedQueries
– Allow updating saved queriesUpdateScheduledReports
– Allow updating scheduled reportsUpdateTriggers
– Allow updating alerts and scheduled searchesDeleteActions
– Allow deleting actionsDeleteDashboards
– Allow deleting dashboardsDeleteFiles
– Allow deleting CSV filesDeleteSavedQueries
– Allow deleting scheduled reportsDeleteScheduledReports
– Allow deleting saved queriesDeleteTriggers
– Allow deleting alerts and scheduled searches
These permissions can currently only be assigned using the LogScale GraphQL API and are not supported in the LogScale UI.
For more information, see Repository & View Permissions.
View permissions to allow for creating different types of assets in a view have been added.
For instance granting a user the
CreateFiles
permission in a view, will allow the user to create new files, but not edit existing files.CreateActions
- Allow creating actionsCreateDashboards
- Allow creating dashboardsCreateSavedQueries
- Allow creating saved queriesCreateScheduledReports
- Allow creating scheduled reportsCreateTriggers
- Allow creating alerts and scheduled searches
These permissions can currently only be assigned using the LogScale GraphQL API.
For more information, see Repository & View Permissions.
Purpose of the repository&view permission
ChangeTriggers
has changed: it is now intended for creating, deleting and updating alerts and scheduled searches. This permission is no longer needed to view alerts and scheduled searches in read-only mode from theAlerts
page: instead, theReadAccess
permission is required for that.For multiple configured SAML IdP certificates, Falcon LogScale now enforces that at least one of them is valid and not expired. This prevents login failures that have occurred due to the expiration of one of the certificates.
For more information, see Certificate Rotation.
Creating roles that have an empty set of permissions is now supported in the
role-permissions.json file
file. To allow this, add the following line to the file:JAVASCRIPT"options": { "allowRolesWithNoPermissions": true }
This ensures compatibility when migrating from previous
view-group-permissions.json
file, should this contain roles without permissions.For more information, see Setting up Roles in a File.
UI Changes
The Time Selector now allows setting advanced relative time ranges that includes both a start and an end, and time anchoring
For more information, see Changing Time Interval, Advanced Time Syntax.
The maximum number of fields that can be added in a Field Aliasing schema has been increased from 50 to 1,000.
The logging for LogScale Multi-Cluster Search network requests have been improved by adding new endpoints that have the
externalQueryId
in the path and thefederationId
in a query parameter.The proxy endpoints for LogScale Multi-Cluster Search have changed. Specific internal marked endpoints that match the external endpoints for proxying are added. This will improve the ability to track multi-cluster searches in the LogScale requests log.
Documentation
The naming structure and identification of release types has been updated. LogScale is available in two release types:
Generally Available (GA) releases — includes new functionality. Using a GA release gets you access to the latest features and functionality.
GA releases are deployed in LogScale SaaS environments.
Long Term Support (LTS) releases — contains the latest features and functionality.
LogScale on-premise customers are advised to install the LTS releases. LTS releases are provided approximately every six weeks.
Security fixes are applied to the last three LTS releases.
GraphQL API
Add a new GraphQL API for getting non-default buckets storage configurations for organizations onDefaultBucketConfigs. The intended use is to help managing a fleet of LogScale clusters.
Field aliases now have API support for being exported and imported as YAML.
Introducing the view field on GraphQL
FileEntry
type, accessible through the entitiesSearch field.The
GA
status has been removed from the following GraphQL mutations:A modifiedInfo field has been added to the following GraphQL types, to provide information about when and by whom the asset was last modified:
If the Enable or Disable actions are used or edited within the UI, the modifiedInfo will also be updated.
Configuration
The new dynamic configuration parameter
ParserBacktrackingLimit
has been added to govern how many new events can be created from a single input event in parsers.This was previously controlled by the
QueryBacktrackingLimit
configuration parameter, which now applies only to queries, thus allowing for finer control.Kafka resets described at Switching Kafka do no longer occur by default. In order to provide safeguard against accidental misconfiguration, the
ALLOW_KAFKA_RESET_UNTIL_TIMESTAMP_MS
environment variable has been added, which per default ensures that Kafka resets are not allowed. With this variable unset, accidental Kafka resets are avoided until an administrator assents to having a Kafka reset performed.To intentionally perform a Kafka reset, administrators should set
ALLOW_KAFKA_RESET_UNTIL_TIMESTAMP_MS
to an epoch timestamp in near future (for instance now + one hour), which will make sure that the setting is automatically disabled again once the reset is complete.For more information, see
ALLOW_KAFKA_RESET_UNTIL_TIMESTAMP_MS
.Mini-segments auto-tune their max block count, up to their limit from configuration. This allows bigger minis for fast datasources, which reduces the number of minis in the global change stream.
Dashboards and Widgets
Improved user experience for creating and configuring dashboards parameters, providing immediate feedback when the setup changes and improved error validation.
Saving changes in parameters settings does not require an additional step to apply the changes before saving the dashboard, making it consistent with saving all other dashboard configurations.
Changes in the Parameters settings side panel now give immediate feedback on the dashboard.
Errors in the parameters setup are now validated on dashboard save, informing users about identified issues.
In the Query Parameter type, the Query String field has been replaced with LogScale Query editor, providing rich query writing experience as well as syntax validation.
In the File Parameter type, additional validation was added to display a warning if the lookup file used as a source of suggestions was deleted.
Parameters have now additional states (error, warning, info) informing users about issues with the setup.
Added the ability to move dashboards parameters to a parameter panel from the configuration side panel.
Added the ability to drag widgets to Sections when in Editing dashboard mode.
Queries
Nested repetitions/quantifiers in the Falcon LogScale Regular Expression Engine v2 are now supported. Nested repetitions are constructions that repeat or quantify another regex expression that contains repetition/quantification. For instance, the regex:
/(?<ipv4>(?:\d{1,3}\.){3}\d{1,3})/
makes use of nested repetitions, namely:
(?:\d{1,3}\.){3}
For more information, see LogScale Regular Expression Engine V2.
Added support for using the new experimental LogScale Regular Expression Engine v2 by specifying the
F
flag, for example:logscale'/foo/F'
The new engine is currently under development and while it can be faster in some cases, there may also be cases where it is slower.
For more information, see LogScale Regular Expression Engine V2.
LogScale Regular Expression Engine v2 now improves optimizer's ability to make alternations into decision trees.
For more information, see LogScale Regular Expression Engine V2.
Introducing a regex backtracking limit of 0,5 seconds pr. input for the Falcon LogScale Regex Engine v2. As soon as the regex starts backtracking to find matches, it is timed and cancelled if the backtracking to find a match exceeds 0.5 seconds. This is done to avoid instances of practically infinite backtracking, as can be the case with some regexes.
For more information, see LogScale Regular Expression Engine V2.
Added optimizations for start-of-text regex expressions with LogScale Regular Expression Engine v2. In particular:
/^X/
and:
/\AX/
no longer try to match all positions in the string.
When doing tests on large body of text, these optimizations have proven to be faster and shown improvements of ~202%, for example when tested against a collection of works by Mark Twain.
For more information, see LogScale Regular Expression Engine V2.
Under the hood changes to how the size of certain events is estimated should now make query state size estimates more realistic.
Query warnings are now included in the activity logs for queries
When a query is rejected due to a validation exception, an activity log is added
Activity logs for queries are now generated for LogScale Self-Hosted
Functions
Introducing the new query function
coalesce()
. This function accepts a list of fields and returns the first value that is not null or empty. Empty values can also be returned by setting a parameter in the function.For more information, see
coalesce()
.Introducing the new query function
array:drop()
. This function drops all consecutive fields of a given array, starting from index 0.For more information, see
array:drop()
.The new
objectArray:eval()
query function is now available for processing structured/nested arrays.For more information, see
objectArray:eval()
.The
array:eval()
query function for processing flat arrays is no longer experimental.For more information, see
array:eval()
.
Fixed in this release
Upgrades
A regression introduced with the upgrade to Java 23 in version 1.158.0 has now been fixed. The issue broke SASL support for Kafka, see Kafka documentation for more information.
UI Changes
The OIDC and SAML configuration pages under Organization settings have been fixed due to a tooltip containing a link that would close before users could click the link.
Entering new arguments for Multi-value Parameters in Dashboard Link Interactions would not actually insert the new argument into the list of arguments. This issue has now been fixed.
Suggestions for parameter values in the Interactions panel would not be able to find fields in the query result. This issue has now been fixed.
A minor UI issue in dropdown windows has been fixed e.g., the Time interval window popping up from the Time Selector would close if any text inside the window fields was selected and the mouse click was released outside the window.
Clean up state for multi-cluster searches that could result in a build up of memory used.
Automation and Alerts
The severity of log message Alert found no results and will not trigger for Aggregate Alerts has been adjusted from
Warning
toInfo
.
Storage
An issue has been fixed which could cause clusters with too few hosts online to reach the configured segment replication factor to run segment rebalancing repeatedly.
The rebalancing now disables itself in such a situation, until enough nodes come back online that rebalancing will actually be able to reach the replication factor.
NullPointerException error occuring since version 1.156.0 when closing segment readers during
redactEvent
processing has now been fixed.
API
An issue has been fixed in the computation of the
digestFlow
property of the query response. The information contained there would be stale in cases where the query started from a cached state or there were digest leadership changes (for example, in case of node restarts).For more information, see Polling a Query Job.
Dashboards and Widgets
The tooltip description of a widget would be cut off if the widget took up the whole row. This issue has now been fixed.
Newline characters would not be escaped in the dashboard parameter input field, thus appearing as not being part of the value. This issue has now been fixed.
Ingestion
When creating a new event forwarding rule, the editor could not be editable in some cases. This issue has now been fixed.
Fixed issues related to searching for ingest timestamp:
Issues with the usage of the query state cache when searching by ingest timestamp.
Reject queries where query time interval starts before the UNIX epoch. This applies both when searching by ingest timestamp or event timestamp. Previously such a query by ingest timestamp would cause an error, but a query by event timestamp was allowed, but not useful as all events in LogScale have event timestamps after the UNIX epoch.
When searching by ingest timestamp,
start()
andend()
functions now report the correct search range.Use event timestamp in place of ingest timestamp if the latter is missing. In old versions of LogScale (prior to 1.15) ingest timestamp was not stored with events. In order to support correct filtering when searching via ingest timestamp also for such old data, LogScale now considers the event timestamp to be also the ingest timestamp.
Log Collector
Fixed a performance issue when sorting by config name in the Fleet Management overview which could result in 503s from the backend.
Queries
Fixed stale QuerySessions that could cause invalid queries to be re-used.
Stopping queries that use early stopping criteria were wrongly reported as
Cancelled
instead ofDone
. The query status has now been fixed.Fixed an issue where non-greedy repetition and repetition of fixed width patterns would not adhere to the backtracking limit in the LogScale Regular Expression Engine V2.
A regression issue that occurred in LogScale version 1.142.0 has now been fixed: it could cause LogScale to exceed the limit on off-heap memory when running many queries concurrently.
Queries hitting the limit on off-heap memory could be deprioritized more strongly than intended. This issue has now been fixed.
Query poll would not be re-tried on dashboards if the request timed out.
Building tables for a query would block other tables from being built due to an internal cache implementation behaviour, which has now been fixed.
Functions
Fixed some cases where
writeJson()
would output fields as numbers that are not supported by the JSON standard. These fields are now represented by strings in the output to ensure that the resulting JSON is valid.A regression issue has been fixed in the
match()
function incidr
mode, which was significantly slower when doing submission of the query.
Other
Off-heap memory limiting might not apply correctly.
A regression issue where some uploaded files close to 2GB could fail to load has now been fixed.
Early Access
Security
It is now possible to map one IdP group name to multiple Falcon LogScale groups during group synchronization. Activate the
OneToManyGroupSynchronization
feature flag for this functionality. With the feature flag enabled, Falcon LogScale will map a group name to all Falcon LogScale groups in the organization that have a matchinglookupName
ordisplayName
, while also performing validation for identical groups. If the multiple mapping feature is not enabled, the existing one-to-one mapping functionality remains unchanged.For more information on how feature flags are enabled, see Enabling & Disabling Feature Flags.
For more information, see Group Synchronization.
Configuration
A new dynamic configuration
AggregatorOutputRowLimit
has been added, along with the new organisation-levelCancelQueriesExceedingAggregateOutputRowLimit
configuration, which is currently under feature flag.Aggregate Query Functions in queries that output more rows than the limit specified by the
AggregatorOutputRowLimit
configuration will get cancelled if theCancelQueriesExceedingAggregateOutputRowLimit
configuration is enabled.These configuration items are being added to allow LogScale administrators to protect the health of the cluster in cases where queries use runaway amounts of resources in the result phase of query execution, impacting cluster health and availability.
For more information, see Dynamic Configuration Parameters.
Improvement
UI Changes
The
Amazon S3 archiving
UI page now correctly points to the S3 Archiving documentation pages versioned for Self-Hosted and Cloud.
Automation and Alerts
The error message The alert query did not start within {timeout}. LogScale will retry starting the query. has been fixed to show the actual timeout instead of just {timeout}.
In the emails sent by email actions, the text
Open in Humio
has been replaced byOpen in LogScale
.
Dashboards and Widgets
Dashboard parameter suggestions of the FixedList Parameter type now follow the order in which they were configured.
Dashboard parameter suggestions of the Query Parameter type now follow the order of the query result.
Ingestion
Data ingest rate monitoring has been adjusted to ensure it reports from nodes across all node roles. Additionally, the number of nodes reporting in large clusters has been raised.
Queries
Some internal improvements have been made to query coordination to make it more robust in certain cases — in particular with failing queries — with an impact on the timing of some API responses.
Some internal improvements have been made to query caching and cache distribution.
The enforcement of the limit on off-heap buffers for segments being queried has been tightened: the limit should no longer exceed the size required for reading a single segment, even in cases where the scheduler is very busy.