Falcon LogScale 1.88.0 LTS (2023-05-24)

Version?Type?Release Date?Availability?End of Support

Security

Updates

Upgrades

From?

Config.

Changes?
1.88.0LTS2023-05-24

Cloud

2024-05-31No1.44.0Yes

Hide file hashes

Show file hashes

Download: https://repo.humio.com/repository/maven-releases/com/humio/server/1.88.0/server-1.88.0.tar.gz

Bug fixes and updates.

Behavior Changes

Scripts or environment which make use of these tools should be checked and updated for the new configuration:

  • Storage

    • It is no longer allowed for nodes to delete bucketed mini-segments involved in queries off local disks before the queries are done. This should help ensure queries do not "miss" querying these files if they are deleted while a query is running.

    • Change how downloads from bucket storage are prioritized for queries. Previously the highest priority query was allowed to download as many segments as it liked. We now try to estimate how much work a query has available in local segments, and prioritize fetching segments for those queries that are close to running out of local work and becoming blocked for that reason.

Upgrades

Changes that may occur or be required during an upgrade.

  • Other

    • Docker images have been upgraded to Java 19.0.2. to address CVE-2022-45688 issue.

    • Snakeyamls has been upgraded to 2.0 to address CVE-2022-1471 issue.

New features and improvements

  • UI Changes

  • Automation and Alerts

    • Clicking the Labels button in Alerts will now show every unique label that has been created on every alert in the same repository. This means that you don't need to rewrite a label when wanting to add the same label to another alert. This feature also applies to Scheduled Searches.

    • The error message for an Alert or Scheduled Search on their edit pages now has a button for clearing the error while the dismiss icon will just close the message but not clear errors.

    • When creating a new Alert, you now have a pulldown menu that suggests labels that you've previously created for other alerts. The same applies to Scheduled Searches.

      For more information, see Creating Alerts.

    • The default time window for Alerts has been updated:

      • When creating an alert from the Alerts page, the default query time window has been changed from 24 Hours to 1 Hours to match the default throttle time.

      • When creating an alert from the Search page, the default Throttle period has been changed to match that of the query time window set.

      For more information, see Creating Alerts.

    • When enabling an Alert or Scheduled Search with no actions, an inline warning message now appears instead of a message box.

  • GraphQL API

  • Configuration

    • New configuration parameters have been added allowing control of client.rack for our Kafka consumers:

      • KAFKA_CLIENT_RACK_ENV_VAR — this variable is read to find the name of the variable that holds the value. It defaults to ZONE, which is the same variable applied to the LogScale node zones by default.

    • Using the storage class "S3 Intelligent-Tiering" in AWS S3 selectively on files that LogScale knows continues to be supported: it is controlled by the new dynamic configuration BucketStorageUploadInfrequentThresholdDays that sets the minimum number of days of remaining retention for the data in order to switch from the default "S3 Standard" to the "Intelligent" tier.

      The decision is made at the point of upload to the bucket only, whereas existing objects in the bucket are not modified.

      The bucket must be configured to not allow the optional tiers Archive Access tier nor Deep Archive Access tier as those do not have instant access, which is required for LogScale.

      As a consequence of that, do not enable automatic archiving within the S3 Intelligent-Tiering storage class.

    • The new configuration parameter SEGMENT_READ_FADVICE has been introduced.

    • The following cluster-level setting has been introduced, editable via GraphQL mutations:

      This is also configurable via the DEFAULT_SEGMENT_REPLICATION_FACTOR configuration parameter.

      If configured via both environment variable and GraphQL mutation, the mutation has precedence.

      For new clusters the default is 1. For clusters upgrading from older versions, the initial value is taken from the STORAGE_REPLICATION_FACTOR environment variable, if set. If instead the variable is not set, the value is taken from the replication factor of the storage partition table prior to the upgrade — this means that upgrading clusters should see no change to their replication factor, unless specified in the STORAGE_REPLICATION_FACTOR .

      The feature can be disabled in case of problems via either the GraphQL mutation setAllowRebalanceExistingSegments, or the environment variable DEFAULT_ALLOW_REBALANCE_EXISTING_SEGMENTS.

      If you need to disable the feature, please reach out to Support and share your concerns so we can try to address them. We intend to remove the option to handle segment partitions manually in the future.

    • Disable the AutomaticDigesterDistribution feature by default. While the feature works, it can cause performance issues on very large installs if nodes are rebooted repeatedly. In future versions, we've worked around this issue, but for 1.88 patch versions, we prefer simply disabling the feature.

  • Dashboards and Widgets

    • When using the Edit in search view item on a dashboard widget, the values set in parameters in the query are also carried over into the search view.

    • Introduced a new setting for dashboard parameters configuration to defer query execution: the dashboard will not execute any queries on page load until the user provides a value to the parameter.

      For more information, see Configuring Dashboard Parameters.

    • The new interaction type Search Link has been introduced, allowing users to create an interaction that will trigger a new search.

      For more information, see Manage Dashboard Interactions, Creating Event List Interactions.

    • You can now save interactions with a saved query on the Search page. Interactions in saved queries are also supported in Packages.

      For more information, see Creating Event List Interactions.

    • The new interaction type Update Parameters has been introduced. This interaction allows you to update parameters in the context you're working in — on the dashboard or on the Search page.

      For more information, see Update Parameters.

    • The combo box has been updated to show multiple selections as "pills".

    • You can now delete or duplicate Event List Interactions from the Interactions overview page.

      For more information, see Deleting & Duplicating Event List Interactions.

    • Multivalued parameters have been introduced to pass an array of values to the query. The support is limited to the Dashboards page.

      For more information, see Multi-value Parameters.

    • When Setting Up a Dashboard Interaction, the {{ startTime }} and {{ endTime }} special variables now work differently, depending on whether the query, widget or dashboard is running in Live mode or not. They now work as follows:

      • In a live query or dashboard, the startTime variable will contain the relative time, such as 2d whereas endTime will be empty.

      • In a non-live query or dashboard, startTime will be the absolute start time when the query was last run. endTime, similarly, will have the end time of when the query was last run.

    • Interactive elements in visualizations now have the point cursor.

  • Log Collector

    • On the Config Overview page a column showing the state of the configuration has been added. The configuration can either be published or in draft state.

      A menu item has been added on the Config Overview page, that links to the Settings page.

      When clicking on an Error status on the Fleet Overview page, a dialog with the error details will open.

      For more information, see Falcon Log Collector Manage your Fleet.

    • Fleet Management updates:

      • Added the Basic Information page with primary information of a specific configuration e.g. name, description, no. of assigned instances.

      • The Config Editor used to create/modify LogScale Collector configurations in LogScale has been augmented with context aware auto-completion, tooltips for keywords and highlighting of invalid settings.

      For more information, see Manage Remote Configurations.

  • Queries

    • Reduced the amount of memory used when multiple queries use the match() function with the same arguments. Before, if you ran many queries that used the same file, the contents of the file would be represented multiple times in memory, once for each query. This could put you at risk of exhausting the server's memory if the files were large. With this change the file contents will be shared between the queries and represented only once. This enables the server to run more queries and/or handle larger files.

      For more information, see Lookup Files Operations.

    • Improvements to query scheduler logic for "shelving" i.e., pausing queries considered too expensive. The pause/unpause logic are now more responsive and unpause queries faster when they become eligible to run.

  • Functions

    • Performance improvements have been made to the match() query function in cases where ignoreCase=true is used together with either mode=cidr, or mode=string.

    • base64Decode() query function has been updated such that, when decoding to UTF-8, invalid code points are replaced with a placeholder character.

    • When IOCs are not available, the ioc:lookup() query function will now produce an error. Previously, it only produced a warning.

    • The memory usage of the functions selectLast() and groupBy() has been improved.

  • Other

    • When the automatic segment rebalancing feature is enabled, ignore the segment storage table when evaluating whether dead ephemeral nodes can be removed automatically.

    • Create Repositories permission now also allows LogScale Self-Hosted users to create repositories.

  • Packages

    • The size limit of packages' lookup files has been changed to adhere to the MAX_FILEUPLOAD_SIZE configuration parameter. Previously the size limit was 1MB.

      For more information, see Exporting the Package.

Fixed in this release

  • UI Changes

    • The Search page would reload when using the browser's history navigation buttons. This issue has now been fixed.

    • An issue in the Usage page that could fail showing any data has been fixed.

      The Usage page now shows an error if there are any warnings from the query.

    • The Fields Panel flyout displayed the bottom 10 values rather than the top 10 values. This issue has now been fixed.

      For more information, see Displaying Fields.

  • Dashboards and Widgets

    • "" was being discarded when creating URLs for interactions. This issue has now been fixed.

    • Attempting to remove a widget on a dashboard would sometimes remove another widget than the one attempted to remove. This issue has been fixed.

    • The tooltip in the Time Chart widget would not show any data points. This issue has now been fixed.

    • Non-breaking space chars (ALT+Space) made Template Expressions unable to be resolved. This issue has been fixed.

    • '_' was not recognized as a valid first symbol for parameters when parsing queries. This issue has now been fixed.

    • Fixed an issue where clicking the Inspect link in alert notifications would land on a missing page.

    • The values of FixedList Parameter on a dashboard would change sort ordering after being exported to a yaml template file. This issue has been fixed.

  • Queries

    • In clusters with bucket storage running queries that take more than 90 minutes, those queries could spuriously fail with a complaint that segments were missing. The issue has now been fixed.

    • Export query result to file dialog would not close in some cases. This issue has now been fixed.

    • Restart of queries based on lookup files has been fixed: only live queries need restarting from changes to uploaded files that they depend on. Scheduled Searches and static queries use the version of the file present when they start and run to completion.

  • Functions

  • Other

    • An issue that would cause query workers to handle mini-segments for longer than intended has been fixed.

    • The following audit log issues have been fixed:

      • the audit log logged the name of the view owning the view bindings instead of the repository it links to. The name now matches the id in the binding log entry.

      • the audit log for a view update did not use the updated view but the view data before the update.

    • An uploaded file would sometimes disappear immediately after uploading. This issue has been fixed.

    • An issue that would cause bucket downloads to retry infinitely many times for certain types of segments has been fixed.

    • Fixed bucket downloads that could fail if the segment they were fetching disappeared from global.

    • In ephemeral-disk mode, allow removing a node via the UI when it is dead regardless of any data present on the node: ephemeral mode knows how to ensure durability also when nodes are lost without notice.

      For more information, see Ephemeral Nodes and Cluster Identity.