Falcon LogScale 1.112.3 LTS (2024-01-30)

Version?Type?Release Date?Availability?End of Support

Security

Updates

Upgrades

From?

Config.

Changes?
1.112.3LTS2024-01-30

Cloud

2024-11-30No1.70.0No

Hide file hashes

Show file hashes

Download: https://repo.humio.com/repository/maven-releases/com/humio/server/1.112.3/server-1.112.3.tar.gz

These notes include entries from the following previous releases: 1.112.1, 1.112.2

Bug fixes and updates.

Advance Warning

The following items are due to change in a future release.

  • Installation and Deployment

    • We intend to drop support for Java 17, making Java 21 the minimum. We plan to make this change in March 2024.

Removed

Items that have been removed as of this release.

Installation and Deployment

  • All ZooKeeper-related functionality for LogScale was deprecated in December 2022, and is now removed:

    • Removed the ZooKeeper status page from the User Interface

    • Removed the ZooKeeper related GraphQL mutations

    • Removed the migration support for node IDs created by ZooKeeper, as we no longer support upgrading from version prior to 1.70.

    Depending on your chosen Kafka deployment, ZooKeeper may still be required to support Kafka.

  • Running on Java 11, 12, 13, 14, 15 and 16 is no longer supported. The minimum supported Java version is 17 starting from this LogScale release.

Storage

  • The unused humio-backup symlink inside Docker containers has been removed.

GraphQL API

  • The deprecated client mutation ID concept is now being removed from the GraphQL API:

    • Removed the clientMutationId argument for a lot of mutations.

    • Removed the clientMutationId field from the returned type for a lot of mutations.

    • Renamed the ClientMutationID datatype, that was returned from some mutations to BooleanResultType datatype. Removed the clientMutationId field on the returned type and replaced it by a boolean field named result.

  • Most deprecated queries, mutations and fields have now been removed from the GraphQL API.

Configuration

Deprecation

Items that have been deprecated and may be removed in a future release.

  • The following REST endpoints for deleting events have been deprecated:

    • /api/v1/dataspaces/(Id)/deleteevents

    • /api/v1/repositories/(id)/deleteevents

    The new GraphQL mutation redactEvents should be used instead.

Behavior Changes

Scripts or environment which make use of these tools should be checked and updated for the new configuration:

  • Automation and Alerts

    • We have changed how Standard Alerts handle query warnings. Previously, LogScale only triggered alerts if there were no query warnings. Now, alerts will trigger despite most query warnings, and the alert status will show a warning instead of an error. Up until now, all query warnings were treated as errors. This meant that the alert did not trigger even though it produced results, and the alert was shown with an error in LogScale. Most query warnings mean that not all data was queried. The previous behaviour prevented the alert from triggering in cases where it would not have, if all data had been available. For instance, an alert that would trigger if a count of events dropped below a threshold. On the other hand, it made some alerts not trigger, even though they would still have if all data was available. That meant that previously you would almost never get an alert that you should not have gotten, but you would sometime not get an alert that you should have gotten. We have reverted this. With this change, we no longer recommend to set the configuration option ALERT_DESPITE_WARNINGS to true, since it treats all query warnings as non-errors, and there are a few query warnings that should make the alert fail.

      For more information, see Diagnosing Alerts.

Upgrades

Changes that may occur or be required during an upgrade.

  • Security

  • Configuration

    • Docker containers have been upgraded to Java 21.

New features and improvements

  • Installation and Deployment

    • Configure LogScale to write fatal JVM error logs in the JVM logging directory, which is specified using JVM_LOG_DIR variable. The default directory is /logs/humio.

  • UI Changes

    • Most tables inside the LogScale UI now supports resizing columns, except the Table widget used during search.

    • The behavior of the ComboBox has changed: the drop-down is not filtered until the text in the filter field has been edited, allowing you to easily copy, alter or clear the text.

    • The list of permissions now has a specific custom order in the UI, as follows.

      • Organization:

        1. Organization settings

        2. Repository and view management

        3. Permissions and user management

        4. Fleet management

        5. Query monitoring

        6. Other

      • Cluster management:

        1. Cluster management

        2. Organization management

        3. Subdomains

        4. Others

    • A combined view of permissions is now available to show all roles listed together when there is more than one role under each repository, organization, or system.

      For more information, see Aggregate Permissions.

    • It is now possible to highlight results based on the filters applied in queries. This helps significantly when trying to understand why a query matches the results or when looking for a specific part of the events text.

      For more information, see Filter Match Highlighting.

  • Automation and Alerts

    • The new button Import from has been added to the Scheduled Searches form allowing importing a Scheduled Search from template or package.

    • When creating or updating Scheduled Searches using the GraphQL API, it is now possible to refer to actions in Packages using a qualified name of \"packagescope/packagename:actionname\". Actions in packages will no longer be found if using an unqualified name.

    • When generating CSV files for attaching to emails or uploading to LogScale in actions, or when using the message template {events_html}, the field @ingesttimestamp is now formatted similar to how @timestamp is.

    • The UI flow for Scheduled Searches has been updated: when you click on New Scheduled Search it will directly go to the New Scheduled Search form.

    • The Alert forms will not show any errors when the alert is disabled.

  • Storage

    • JVM_TMP_DIR has been added to the launcher script. This option is used for configuring java.io.tmpdir and jna.tmpdir for the JVM. The directory will default to jvm-tmp inside the directory specified by the DIRECTORY setting. This default should alleviate issues starting LogScale on some systems due to the /tmp directory being marked as noexec.

      For more information, see Troubleshooting: Error Starting LogScale due to Exec permissions on /tmp.

    • Bucket storage cleaning of tmp files now only runs on a few nodes in the cluster rather than on all nodes.

  • GraphQL API

    • Added limits for GraphQL queries on the total number of selected fields and fragments. Defaults are 1000 for authenticated and 150 for unauthenticated users.

      Cluster administrators can adjust these limits with the GraphQLSelectionSizeLimit and UnauthenticatedGraphQLSelectionSizeLimit dynamic configurations.

    • The contentHash field on the File output type has been reintroduced.

  • Configuration

    • LOCAL_STORAGE_PREFILL_PERCENTAGE new configuration option has been added.

      For more information, see LOCAL_STORAGE_PREFILL_PERCENTAGE.

    • Query queueing based on the available memory in query coordinator is enabled by default by treating dynamic configuration QueryCoordinatorMaxHeapFraction as 0.5, if it has not been set. To disable queing, set QueryCoordinatorMaxHeapFraction to 1000.

    • Set the default value of LOCAL_STORAGE_PERCENTAGE to 85, and the minimum value to 0. The default was previously to leave this unset, which is not safe in clusters where bucket storage contains more data than will fit on local drives.

    • The new environment variable DISABLE_BUCKET_CLEANING_TMP_FILES has been introduced. It allows to reduce the amount of listing of tmp files in bucket.

  • Ingestion

    • When writing parsers, the fields produced by a test case are now available for autocompletion in the editor.

      For more information, see Using the Parser Code Editor.

  • Dashboards and Widgets

    • You can enable the export of Dashboards to a PDF file, with many options available to control the output layout and formatting.

      The feature is available to all users who already have access to dashboard data. This is the first of two feature releases, aiming to provide full schedulable PDF reporting capabilities to LogScale.

      For more information, see Export Dashboards as PDF.

    • The new Gauge widget is introduced: it allows you to represent values on a fixed scale, offering a visual and intuitive way to monitor key performance metrics.

      For more information, see Gauge Widget.

    • A parameter configuration option has been added to support invalidation of parameter inputs. The format for this is a comma separated list of invalid input patterns (regexes).

    • Introduced a new style option Show 'Others' to the Time Chart Widget: it allows you to show/hide other series when there are more series than the maximum allowed in the chart.

    • A parameter configuration option has been added to allow setting a custom message when a parameter input is invalid.

    • New formatting options have been introduced for the Table widget, to get actionable insights from your data faster:

      • Conditional formatting of table cells

      • Text wrapping and column resizing

      • Row numbering

      • Number formatting

      • Link formatting

      • Columns hiding

      For more information, see Table Widget.

  • Log Collector

    • The Fleet Management tab on Fleet Overview page is now renamed to Data Ingest.

  • Functions

  • Packages

    • Filter alerts and Standard alerts are now shown in the same tab Alerts under Assets when installing or viewing installed Packages.

    • It is now possible to see the type of action in Packages (Marketplace, Installed and Create a package).

Fixed in this release

  • UI Changes

    • Queries could "flicker" for a short period causing "negative alerts" to trigger for no reason (negative alerts are alerts that check for the absence of events). This issue has been fixed.

    • The following issue has been fixed on the Search page: if regular expressions contained named groups with special characters (underscore _ for example) a recent change with the introduction of Filter Match Highlighting would cause a server error and hang the UI.

    • The following items about Saving Queries have been fixed:

      • The Search... field for saved queries did not return what would be expected.

      • Upon reopening the Queries dropdown after having filled out the Search... field, the text would still be present in the Search... field but not filter on the queries.

      • Added focus on the Search... field when reopening the Queries dropdown.

  • Automation and Alerts

    • Notifications on problems with Filter Alerts where not automatically removed when the problem was solved. This issue is now fixed.

    • Filter alerts that could fail right after a cluster restart have now been fixed.

    • When used with Filter Alerts, the {events_html} message template would not keep the order of the fields from the Alert query.

  • Storage

    • A workaround solution has been identified for those cases where segment files on local disk no longer pass their internal checksum test and are detected as "broken" by the background merge process.

      1. Ensure a copy of the local file is present in the bucket storage, backing up the cluster

      2. Delete the local copy

      As a result, any merge attempt involving that file will succeed after the next restart of LogScale.

  • GraphQL API

    • When trying to delete an Alert, Scheduled Search or Dashboard using a mutation for one of the other types, it would end up in a state where it was not deleted, but could not run either. This issue is now fixed.

  • Ingestion

    • The buttons used for editing and deleting an ingest listener were overlapping in Safari on the Ingest Listeners page under a repository. This issue has been fixed.

  • Dashboards and Widgets

    • Field values containing % would not be resolved correctly in interactions. This issue has been fixed.

  • Functions

    • Results for empty buckets didn't include the steps after the first aggregator of the subquery. This issue has now been fixed.

    • match() function using a json file and containing an object with a missing field, could lead to an internal error.

    • The regex() function has been fixed for cases where \Q...\E could cause problems for named capturing groups.

    • The array:filter() function has been fixed for an issue that caused incorrect output element values in certain circumstances.

  • Other

    • A cluster with very little disk space left could result in excessive logging from com.humio.distribution.RendezvousSegmentDistribution.

    • Fixing a race that can leave a query in a state where it will cause an excessive amount of 404 HTTP requests. This adds unnecessary noise and a bit of extra load to the system.

    • A minor logging issue has been fixed: ClusterHostAliveStats would log that hosts were "changed from being considered dead to alive" on hosts that had just rebooted, when such hosts actually consider all other nodes alive for a little while, to allow the booting node some time to hear heartbeats from others.

    • A boot-time version checking issue could cause LogScale to crash on boot, if joining a fresh cluster, and the first node to join that cluster would crash.

  • Packages

    • Updating of a Package failed when using anything other than a personal user token. This issue has been fixed.

    • Updating a package with a lookup file and a parser/scheduled search/filter alert/alert containing match would fail if the new column parameter did not exist in the old lookup file. This issue has now been fixed.

    • Aligned the requirements to allow all tokens (with the correct permissions) to install and update Packages.

    • Fixed a broken link from saved query asset in Packages to Search page.

    • The alert types in Package Marketplace were showing twice — this is now fixed so it properly shows one type as expected.