Falcon LogScale 1.159.1 LTS (2024-10-31)

Version?Type?Release Date?Availability?End of Support

Security

Updates

Upgrades

From?

Config.

Changes?
1.159.1LTS2024-10-31

Cloud

2025-10-31Yes1.112No

Hide file hashes

Show file hashes

Download

Bug fixes and updates.

Deprecation

Items that have been deprecated and may be removed in a future release.

  • The following GraphQL mutations and field have been deprecated, since the starring functionality is no longer in use for alerts and scheduled searches:

  • The lastScheduledSearch field from the ScheduledSearch datatype is now deprecated and planned for removal in LogScale version 1.202. The new lastExecuted and lastTriggered fields have been added to the ScheduledSearch datatype to replace lastScheduledSearch.

  • The deprecated JDK-less server.tar.gz tarball release is no longer being published. Users should switch to either server-linux_x64.tar.gz or server-alpine_x64.tar.gz depending on their operating system.

Behavior Changes

Scripts or environment which make use of these tools should be checked and updated for the new configuration:

  • Automation and Alerts

  • Storage

    • LogScale now avoids moving mini-segments to follow the digest nodes if the mini-segments are available in Bucket Storage. Instead, mini-segments will now be fetched as needed, when the digest leader is ready to merge them. This reduces the load on Global Database in some cases following a digest reassignment.

    • During digest reassignment, LogScale will now ignore mini-segments in Bucket Storage when deciding whether to switch merge targets because some mini-segments are not present locally. This should slightly reduce the load on Global Database during digest reassignment.

    • Allow live query updates to run on a new thread pool digestLive, but only for datasources that spend more time on these updates than allowed in the digester pool on live queries, or for many datasources, if their total load exceeds time available for digesters. This frees up time for the digesters, provided there is available CPU on the node.

    • LogScale now avoids moving merge targets to the digest leader during digest reassignment if those segments are already in Bucket Storage.

  • Ingestion

    • Falcon LogScale now improves decision-making around which segments a digest leader fetches as part of taking over leadership. This should reduce the incidence of small bits of data being replayed from Kafka unnecessarily, and may also reduce how often reassignment will trigger a restart of live queries.

      For more information, see Ingestion: Digest Phase.

  • Queries

    • When a digest node is unavailable, a warning is not attached to queries, but the queries are allowed to proceed.

      This way, the behaviour of a query is similar to the case where a segment cannot be searched, due to all the owning nodes being unavailable at the time of the query.

Upgrades

Changes that may occur or be required during an upgrade.

  • Installation and Deployment

    • The JDK has been upgraded to 23.0.1

    • Bundled JDK is now upgraded to Java 23.

    • Upgraded the Kafka clients to 3.8.0.

New features and improvements

  • Security

    • New view permissions have been added to allow for updating and deleting different types of assets in a view. For instance, granting a user the UpdateFiles permission in a view will allow the user to update files, but not delete or create files.

      View permissions added:

      These permissions can currently only be assigned using the LogScale GraphQL API and are not supported in the LogScale UI.

      For more information, see Repository & View Permissions.

    • View permissions to allow for creating different types of assets in a view have been added.

      For instance granting a user the CreateFiles permission in a view, will allow the user to create new files, but not edit existing files.

      These permissions can currently only be assigned using the LogScale GraphQL API.

      For more information, see Repository & View Permissions.

    • For multiple configured SAML IdP certificates, Falcon LogScale now enforces that at least one of them is valid and not expired. This prevents login failures that have occurred due to the expiration of one of the certificates.

      For more information, see Certificate Rotation.

    • Purpose of the repository&view permission ChangeTriggers has changed: it is now intended for creating, deleting and updating alerts and scheduled searches. This permission is no longer needed to view alerts and scheduled searches in read-only mode from theAlerts page: instead, the ReadAccess permission is required for that.

    • Creating roles that have an empty set of permissions is now supported in the role-permissions.json file file. To allow this, add the following line to the file:

      JAVASCRIPT
      "options": { "allowRolesWithNoPermissions": true
}

      This ensures compatibility when migrating from previous view-group-permissions.json file, should this contain roles without permissions.

      For more information, see Setting up Roles in a File.

  • UI Changes

    • The Time Selector now allows setting advanced relative time ranges that includes both a start and an end, and time anchoring

      For more information, see Changing Time Interval, Advanced Time Syntax.

    • The maximum number of fields that can be added in a Field Aliasing schema has been increased from 50 to 1,000.

    • The logging for LogScale Multi-Cluster Search network requests have been improved by adding new endpoints that have the externalQueryId in the path and the federationId in a query parameter.

    • The proxy endpoints for LogScale Multi-Cluster Search have changed. Specific internal marked endpoints that match the external endpoints for proxying are added. This will improve the ability to track multi-cluster searches in the LogScale requests log.

  • Documentation

    • The naming structure and identification of release types has been updated. LogScale is available in two release types:

      • Generally Available (GA) releases — includes new functionality. Using a GA release gets you access to the latest features and functionality.

        GA releases are deployed in LogScale SaaS environments.

      • Long Term Support (LTS) releases — contains the latest features and functionality.

        LogScale on-premise customers are advised to install the LTS releases. LTS releases are provided approximately every six weeks.

        Security fixes are applied to the last three LTS releases.

  • GraphQL API

  • Configuration

    • The new dynamic configuration parameter ParserBacktrackingLimit has been added to govern how many new events can be created from a single input event in parsers.

      This was previously controlled by the QueryBacktrackingLimit configuration parameter, which now applies only to queries, thus allowing for finer control.

    • Kafka resets described at Switching Kafka do no longer occur by default. In order to provide safeguard against accidental misconfiguration, the ALLOW_KAFKA_RESET_UNTIL_TIMESTAMP_MS environment variable has been added, which per default ensures that Kafka resets are not allowed. With this variable unset, accidental Kafka resets are avoided until an administrator assents to having a Kafka reset performed.

      To intentionally perform a Kafka reset, administrators should set ALLOW_KAFKA_RESET_UNTIL_TIMESTAMP_MS to an epoch timestamp in near future (for instance now + one hour), which will make sure that the setting is automatically disabled again once the reset is complete.

      For more information, see ALLOW_KAFKA_RESET_UNTIL_TIMESTAMP_MS.

    • Mini-segments auto-tune their max block count, up to their limit from configuration. This allows bigger minis for fast datasources, which reduces the number of minis in the global change stream.

  • Dashboards and Widgets

    • Improved user experience for creating and configuring dashboards parameters, providing immediate feedback when the setup changes and improved error validation.

      • Saving changes in parameters settings does not require an additional step to apply the changes before saving the dashboard, making it consistent with saving all other dashboard configurations.

      • Changes in the Parameters settings side panel now give immediate feedback on the dashboard.

      • Errors in the parameters setup are now validated on dashboard save, informing users about identified issues.

      • In the Query Parameter type, the Query String field has been replaced with LogScale Query editor, providing rich query writing experience as well as syntax validation.

      • In the File Parameter type, additional validation was added to display a warning if the lookup file used as a source of suggestions was deleted.

      • Parameters have now additional states (error, warning, info) informing users about issues with the setup.

    • Added the ability to move dashboards parameters to a parameter panel from the configuration side panel.

    • Added the ability to drag widgets to Sections when in Editing dashboard mode.

  • Queries

    • Nested repetitions/quantifiers in the Falcon LogScale Regular Expression Engine v2 are now supported. Nested repetitions are constructions that repeat or quantify another regex expression that contains repetition/quantification. For instance, the regex: 

      /(?<ipv4>(?:\d{1,3}\.){3}\d{1,3})/

      makes use of nested repetitions, namely: 

      (?:\d{1,3}\.){3}

      For more information, see LogScale Regular Expression Engine V2.

    • Added support for using the new experimental LogScale Regular Expression Engine v2 by specifying the F flag, for example:

      logscale Syntax
      '/foo/F'

      The new engine is currently under development and while it can be faster in some cases, there may also be cases where it is slower.

      For more information, see LogScale Regular Expression Engine V2.

    • LogScale Regular Expression Engine v2 now improves the optimizer ability to make alternations into decision trees.

      For more information, see LogScale Regular Expression Engine V2.

    • Introducing a regex backtracking limit of 0,5 seconds pr. input for the Falcon LogScale Regex Engine v2. As soon as the regex starts backtracking to find matches, it is timed and cancelled if the backtracking to find a match exceeds 0.5 seconds. This is done to avoid instances of practically infinite backtracking, as can be the case with some regexes.

      For more information, see LogScale Regular Expression Engine V2.

    • Added optimizations for start-of-text regex expressions with LogScale Regular Expression Engine v2. In particular: 

      /^X/

      and: 

      /\AX/

      no longer try to match all positions in the string.

      When doing tests on large body of text, these optimizations have proven to be faster and shown improvements of ~202%, for example when tested against a collection of works by Mark Twain.

      For more information, see LogScale Regular Expression Engine V2.

    • Under the hood changes to how the size of certain events is estimated should now make query state size estimates more realistic.

      • Query warnings are now included in the activity logs for queries

      • When a query is rejected due to a validation exception, an activity log is added

      • Activity logs for queries are now generated for LogScale Self-Hosted

  • Functions

    • Introducing the new query function coalesce(). This function accepts a list of fields and returns the first value that is not null or empty. Empty values can also be returned by setting a parameter in the function.

      For more information, see coalesce().

    • Introducing the new query function array:drop(). This function drops all consecutive fields of a given array, starting from index 0.

      For more information, see array:drop().

    • The new objectArray:eval() query function is now available for processing structured/nested arrays.

      For more information, see objectArray:eval().

    • The array:eval() query function for processing flat arrays is no longer experimental.

      For more information, see array:eval().

Fixed in this release

  • UI Changes

    • The OIDC and SAML configuration pages under Organization settings have been fixed due to a tooltip containing a link that would close before users could click the link.

    • Entering new arguments for Multi-value Parameters in Dashboard Link would not actually insert the new argument into the list of arguments. This issue has now been fixed.

    • Suggestions for parameter values in the Interactions panel would not be able to find fields in the query result. This issue has now been fixed.

    • A minor UI issue in dropdown windows has been fixed e.g., the Time interval window popping up from the Time Selector would close if any text inside the window fields was selected and the mouse click was released outside the window.

    • Clean up state for multi-cluster searches that could result in a build up of memory used.

  • Automation and Alerts

    • The severity of log message Alert found no results and will not trigger for Aggregate Alerts has been adjusted from Warning to Info.

  • Storage

    • An issue has been fixed which could cause clusters with too few hosts online to reach the configured segment replication factor to run segment rebalancing repeatedly.

      The rebalancing now disables itself in such a situation, until enough nodes come back online that rebalancing will actually be able to reach the replication factor.

    • NullPointerException error occurring since version 1.156.0 when closing segment readers during redactEvent processing has now been fixed.

    • A regression introduced with the upgrade to Java 23 in version 1.158.0 has now been fixed. The issue broke SASL support for Kafka, see Kafka documentation for more information.

  • API

    • An issue has been fixed in the computation of the digestFlow property of the query response. The information contained there would be stale in cases where the query started from a cached state or there were digest leadership changes (for example, in case of node restarts).

      For more information, see Polling a Query Job.

  • Ingestion

    • When creating a new event forwarding rule, the editor could not be editable in some cases. This issue has now been fixed.

    • Fixed issues related to searching for ingest timestamp:

      • Issues with the usage of the query state cache when searching by ingest timestamp.

      • Reject queries where query time interval starts before the UNIX epoch. This applies both when searching by ingest timestamp or event timestamp. Previously such a query by ingest timestamp would cause an error, but a query by event timestamp was allowed, but not useful as all events in LogScale have event timestamps after the UNIX epoch.

      • When searching by ingest timestamp, start() and end() functions now report the correct search range.

      • Use event timestamp in place of ingest timestamp if the latter is missing. In old versions of LogScale (prior to 1.15) ingest timestamp was not stored with events. In order to support correct filtering when searching via ingest timestamp also for such old data, LogScale now considers the event timestamp to be also the ingest timestamp.

  • Dashboards and Widgets

    • The tooltip description of a widget would be cut off if the widget took up the whole row. This issue has now been fixed.

    • Newline characters would not be escaped in the dashboard parameter input field, thus appearing as not being part of the value. This issue has now been fixed.

  • Log Collector

    • Fixed a performance issue when sorting by config name in the Fleet Management overview which could result in 503s from the backend.

  • Queries

    • Fixed stale QuerySessions that could cause invalid queries to be re-used.

    • Stopping queries that use early stopping criteria were wrongly reported as Cancelled instead of Done. The query status has now been fixed.

    • Fixed an issue where non-greedy repetition and repetition of fixed width patterns would not adhere to the backtracking limit in the LogScale Regular Expression Engine V2.

    • A regression issue that occurred in LogScale version 1.142.0 has now been fixed: it could cause LogScale to exceed the limit on off-heap memory when running many queries concurrently.

      Queries hitting the limit on off-heap memory could be deprioritized more strongly than intended. This issue has now been fixed.

    • Query poll would not be re-tried on dashboards if the request timed out.

    • Building tables for a query would block other tables from being built due to an internal cache implementation behaviour, which has now been fixed.

  • Functions

    • Fixed some cases where writeJson() would output fields as numbers that are not supported by the JSON standard. These fields are now represented by strings in the output to ensure that the resulting JSON is valid.

    • A regression issue has been fixed in the match() function in cidr mode, which was significantly slower when doing submission of the query.

  • Other

    • Off-heap memory limiting might not apply correctly.

    • A regression issue where some uploaded files close to 2GB could fail to load has now been fixed.

Early Access

  • Security

    • It is now possible to map one IdP group name to multiple Falcon LogScale groups during group synchronization. Activate the OneToManyGroupSynchronization feature flag for this functionality. With the feature flag enabled, Falcon LogScale will map a group name to all Falcon LogScale groups in the organization that have a matching lookupName or displayName, while also performing validation for identical groups. If the multiple mapping feature is not enabled, the existing one-to-one mapping functionality remains unchanged.

      For more information on how feature flags are enabled, see Enabling & Disabling Feature Flags.

      For more information, see Group Synchronization.

  • Configuration

Improvement

  • UI Changes

    • The Amazon S3 archiving UI page now correctly points to the S3 Archiving documentation pages versioned for Self-Hosted and Cloud.

  • Automation and Alerts

    • The error message The alert query did not start within {timeout}. LogScale will retry starting the query. has been fixed to show the actual timeout instead of just {timeout}.

    • In the emails sent by email actions, the text Open in Humio has been replaced by Open in LogScale.

  • Ingestion

    • Data ingest rate monitoring has been adjusted to ensure it reports from nodes across all node roles. Additionally, the number of nodes reporting in large clusters has been raised.

  • Dashboards and Widgets

    • Dashboard parameter suggestions of the FixedList Parameter type now follow the order in which they were configured.

      Dashboard parameter suggestions of the Query Parameter type now follow the order of the query result.

  • Queries

    • Some internal improvements have been made to query coordination to make it more robust in certain cases — in particular with failing queries — with an impact on the timing of some API responses.

    • Some internal improvements have been made to query caching and cache distribution.

    • The enforcement of the limit on off-heap buffers for segments being queried has been tightened: the limit should no longer exceed the size required for reading a single segment, even in cases where the scheduler is very busy.