Falcon LogScale 1.94.0 LTS (2023-07-05)

Version?Type?Release Date?Availability?End of Support

Security

Updates

Upgrades

From?

Config.

Changes?
1.94.0LTS2023-07-05

Cloud

2024-07-31No1.44.0No

Hide file hashes

Show file hashes

Download: https://repo.humio.com/repository/maven-releases/com/humio/server/1.94.0/server-1.94.0.tar.gz

Bug fixes and updates.

Advance Warning

The following items are due to change in a future release.

  • Installation and Deployment

    • Support for running on Java 11, 12, 13, 14, 15 and 16 will be removed by the end of September 2023.

Removed

Items that have been removed as of this release.

API

  • Degrade and deprecate some REST and GraphQL APIs due to the introduction of AutomaticSegmentDistribution and AutomaticDigesterDistribution. The deprecated elements will be removed in a future release, once the upgrade compatibility with version 1.88.0 is dropped. We expect this to be no earlier than September 2023.

    The following REST endpoints are deprecated, as they no longer have an effect and return meaningless results:

    • api/v1/clusterconfig/segments/prune-replicas

    • api/v1/clusterconfig/segments/distribute-evenly

    • api/v1/clusterconfig/segments/distribute-evenly-reshuffle-all

    • api/v1/clusterconfig/segments/distribute-evenly-to-host

    • api/v1/clusterconfig/segments/distribute-evenly-from-host

    • api/v1/clusterconfig/segments/partitions

    • api/v1/clusterconfig/segments/partitions/setdefaults

    • api/v1/clusterconfig/segments/set-replication-defaults

    • api/v1/clusterconfig/partitions/setdefaults

    • api/v1/clusterconfig/ingestpartitions/distribute-evenly-from-host

    • api/v1/clusterconfig/ingestpartitions/setdefaults

    • api/v1/clusterconfig/ingestpartitions (POST only, GET will continue to work)

    The following GraphQL mutations are deprecated, as they no longer have an effect and return meaningless results:

    • startDataRedistribution

    • updateStoragePartitionScheme

    The IngestPartitionScheme mutation is not deprecated, but as it updates state that is overwritten by automation, we recommend against using it — it exists solely to serve as a debugging tool.

    The following GraphQL fields on the cluster object are deprecated, and return meaningless values:

    • ingestPartitionsWarnings

    • suggestedIngestPartitions

    • storagePartitions

    • storagePartitionsWarnings

    • suggestedStoragePartitions

    • storageDivergence

    • reapply_targetSize

    The following fields in the return value of the api/v1/clusterconfig/segments/segment-stats endpoint are deprecated and degraded to always be O:

    • reapply_targetBytes

    • reapply_targetSegments

    • reapply_inboundBytes

    • reapply_inboundSegments

Behavior Changes

Scripts or environment which make use of these tools should be checked and updated for the new configuration:

  • Storage

    • Be less aggressive updating the digest partitions when a node goes offline. When a node goes offline/online, creating a well balanced table can require changes to partitions other than those where the changed node appears. This can cause more digest reassignment that we'd like, so we're changing the behavior of the automation. We'll now only generate optimally balanced tables in reaction to nodes being registered or unregistered from the cluster, and in reaction to the digest replication factor changing. The rest of the time, we'll take the previously generated balanced table as a starting point, and do very minimal node replacements in it to ensure partitions are properly replicated to live nodes.

    • It is no longer allowed for nodes to delete bucketed mini-segments involved in queries off local disks before the queries are done. This should help ensure queries do not "miss" querying these files if they are deleted while a query is running.

    • Metadata on segments in memory is now represented in a manner that requires less memory at runtime after booting. The heap required for global snapshot is in the range 3-6 times the size of the disk, for a cluster with many segments. This change reduces the memory requirements for long retention compared to previous versions. Note that for a short time during boot of a node the memory requirement is closer to 10-15 times the size of the snapshot on disk.

  • Configuration

New features and improvements

  • UI Changes

    • A new tutorial built on a dedicated demo data view is available for environments that do not have access to legacy tutorial based on a sandbox repository.

    • The DeleteRepositoryOrView data permission is now visible in the UI on Cloud environments.

    • The Time Selector now only allows zooming out to approximately 4,000 years.

    • The ChangeRetention data permission is now enabled on Cloud environments.

    • When reaching the default capped output in table() and sort() query functions, a warning now suggests you can set a new value using the limit parameter.

  • Documentation

    • LogScale Kubernetes Reference Architecture new page has been added with LogScale reference architecture description when deploying LogScale using Kubernetes.

    • Regular Expression Syntax new page has been added with extended details of supported regular expression syntax and differences between the LogScale support and other implementations such as Java and Perl.

  • Automation and Alerts

    • The Alert and Scheduled Search jobs no longer produce logs about specific alerts or scheduled searches in the humio repository. The logs are still sent to the humio-activity repository, which in normal setup is also ingested into the humio repository. So before, the logs would normally be duplicated, now they are not. The only difference between the two types of logs, is that the logs from the humio-activity repository all have loglevel equal to INFO. You can use the severity field instead to distinguish between the severity of the logs.

    • The possibility to mark alerts and scheduled searches as favorites has been removed.

    • Improvements in the layout of Alerts and Scheduled Searches, which now have updated forms.

    • The Actions overview now has quick filters for showing only actions of specific types.

    • The Scheduled Searches overview now shows the status of scheduled searches with a colored dot to make it easy to spot failing scheduled searches.

    • Improvements in the Alerts and Scheduled Searches permissions, which are now renamed to Run on behalf of, and have a more clarifying help text.

    • The Alerts overview now has quick filters for showing only standard alerts or filter alerts. It also shows the status of alerts with a colored dot to make it easy to spot failing alerts.

  • GraphQL API

    • The Usage page has been updated to support queries that are in progress for longer than the GraphQL timeout allows.

    • The semantics of the field SolitarySegmentSize on the ClusterNode datatype has changed from counting bytes that only exist on that node and which have been underreplicated for a while, to counting bytes that only exist on that node.

    • The GraphQL schema for UsageStats has been updated to reflect that queries can be in progress.

    • Mutations enableAlert and disableAlert have been added for enabling and disabling an alert without changing other fields.

  • Configuration

  • Ingestion

    • Parser timeouts have been changed to take thread time into account. This should make parsers more resilient to long Garbage Collector stalls.

      For more information, see Parser Timeout.

  • Dashboards and Widgets

    • New parsing of Template Expressions has been implemented in the UI for improved performance.

    • When creating or editing interactions you can now visualize any unused parameter bindings, with the option to remove them.

      For more information, see Unused parameters bindings.

    • Improved performance on the Search page, especially when events contain large JSON objects.

      A new limit of 49 series has been set when using the wide format data (one field per series) in the Scatter Chart Widget (the first field is always the x axis). No such limit applies to long format data (series defined by one groupby column).

    • The empty list alias is now available as an input option for parameter bindings, so that Multi-value Parameters can be set explicitly to have the value of an empty list.

      For more information, see Empty list alias.

    • Parameter labels are now used instead of parameter IDs when displaying the list of parameters that a widget / query is waiting on.

  • Log Collector

    • Added a new test status for configurations, which allows you to try out a configuration on one or more instances before it's published.

      For more information, see Test a Remote Configuration.

  • Functions

    • Performance improvements when using regex() function or regex syntax.

    • In parseTimestamp() function, special format specifiers like seconds are now recognized independently of capitalization to allow case-insensitive match.

  • Other

    • Reduced the amount of memory used when multiple queries use the match() function with the same arguments. Before, if you ran many queries that used the same file, the contents of the file would be represented multiple times in memory, once for each query. This could put you at risk of exhausting the server's memory if the files were large. With this change the file contents will be shared between the queries and represented only once. This enables the server to run more queries and/or handle larger files.

      For more information, see Lookup Files Operations.

    • When the Kafka broker set changes at runtime, track that set and use as bootstrap servers for Kafka whenever LogScale needs to create a new Kafka client at runtime. This allows replacing all Kafka brokers (incrementally, moving their work to new servers) without restarting LogScale. Note that the set is not persisted across restart of LogScale, so when restarting LogScale, make sure to provide an up to date set of bootstrap servers.

    • The following cluster management features are now enabled:

      • AutomaticJobDistribution

      • AutomaticDigesterDistribution

      • AutomaticSegmentDistribution

      For more information, see Digest Rules.

Fixed in this release

  • UI Changes

    • Turned off the light bulb in the query editor as it was causing technical issues.

    • Fixed an issue where the filter would remain applied in the saved or recent queries when switching tabs in the Queries menu.

    • Fixed the order of the timezones in the timezone dropdown on the Search and Dashboards pages.

    • An error for lacking permissions that appeared when updating the organization settings has been fixed. Now, if you have permissions to view the Organization Settings page, you can also update information on it.

  • Automation and Alerts

    • The throttle field would be empty when editing an Alert; this issue has now been fixed.

    • Fixed an issue where clicking the Inspect link in Alert notifications would land on a missing page.

    • Fixed an issue that could cause some rarely occurring errors when running alerts to not show up on the alert.

  • Dashboards and Widgets

    • Labels of FixedList Parameter parameters values have been fixed, so that they default to the value instead of rendering empty string.

    • Fixed an issue where certain widget options would be ignored when importing a dashboard template or installing a package.

    • The following issues have been fixed on dashboards:

      • A dashboard would sometimes be perceived as changed on the server even though it was not.

      • Discard unsaved changes would appear when creating and applying new parameters.

    • Fixed the Manage interactions page where Event List Interactions were not scrollable.

    • Fixed a wrong behaviour on the Interactions overview page when creating a new interaction: if the interaction panel was opened, the repository options would dropdown in it instead of in the Create new interaction dialog.

  • Queries

    • An edge case has been fixed where query workers could fail to include mini-segments if the mini-segments were merged at a bad time, causing queries to be missing the data in those segments.

  • Functions

    • The select() function has been fixed as it wasn't preserving tags.

    • The format() has been fixed as the combination of the hexadecimal modifier combined with grouping would not always work.

    • The rename() function would drop the field, if the field and as arguments were identical; this issue has now been fixed.

    • The regex engine has been fixed for issues impacting nested repeats and giving false negatives, as in expressions such as (x{2}:){3}.

  • Other

    • Some merged segments could temporarily be missing from query results right after an ephemeral node reboot. This issue has been fixed.

    • The following Node-Level Metrics that showed incorrect results are now fixed: primary-disk-usage, secondary-disk-usage, cluster-time-skew, temp-disk-usage-bytes.

    • Fixed an issue that could cause segments to appear missing in queries, due to the presence of deleted mini-segments with the same target as live mini-segments.

Early Access

  • Automation and Alerts

    • This release includes filter alerts in Early Access. Filter alerts aim to replace existing alerts for use cases where the query does not contain any aggregates.

      Filter alerts:

      • Trigger on individual events and send notifications per event.

      • Guarantee at-least-once delivery of events to actions, within the limits described below.

      • Currently only support delays (ingest delays + delays in actions) of 1 hour and limit the number of notifications to 15 per minute per alert. Before going out of Public GA, those limits will be raised.

      For more information, see Alerts.