Humio Server 1.25.0 Preview (2021-04-29)

VersionTypeRelease DateEnd of SupportUpgrades FromData MigrationConfig. Changes
1.25.0Preview2021-04-292021-05-201.16.0NoNo
JAR ChecksumValue
MD5aa7d71e3617d71ec800f1e5e898cb328
SHA18e815fd8c1d6a0bc1c22e096e8f82d08d40ea172
SHA2567f26e858df2f1e16a64f44c6ac72948267eb317d884ad1db72b3799adcc95696
SHA5121f447d204ecb3e60c80835e02a4fe8e6c7984728df5b31827b23f2278e00a0862a78dabe8ef01a2295f925a351693bdbb1d649c83465bf0951b040eab466fbad

There is a serious issue affecting larger clusters in this release. The global inconsistency checker job can cause the thread responsible for reading changes from global to hang. It is possible to work around this by disabling the job using RUN_GLOBAL_CONSISTENCY_CHECKER_JOB=false. This is fixed in 1.25.2 (and 1.26.0).

The HEC ingest endpoint will no longer implicitly parse logs using the built-in kv parser. Previously, a log ingested using this endpoint would implicitly be parsed with the kv parser when the supplied event field was given as a string. For instance, this log:

json
{
 "time" : 1537537729.0,
 "event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
 "source" : "/var/log/application.log",
 "sourcetype" : "applog",
 "fields" : { "#env" : "prod" }
 }

Would be parsed, so that the resulting Humio event would contain the field name=webserver.

If you don't wish this behavior to change, you will have to perform this parsing operation explicitly.

When ingesting into the HEC endpoint, you are using an ingest token to authenticate with Humio. If that token does not have an associated parser, all you need to do is assign the kv parser to the token.

If your ingest token already has an assigned parser, you will need to prepend the code of that parser with this code snippet: 

kvParse(@rawstring) | findTimestamp(addErrors=false) |

Dark Mode is a new visual theme throughout Humio (except some settings pages) that is tailored to offer great readability in dark environments, to not brighten the entire room when used on dashboards, and offer a unique visual style that some users prefer simply for its aesthetics. In 1.25 users are going to see a modal dialogue that asks what mode users would like to have; dark mode, light mode or follow the OS theme. This setting can later be changed in the settings menu.

Improvements, new features and functionality

  • Other

    • The query scheduler now prioritizes new queries started by a user based on the cumulative cost of recent queries started by that user. Added new configuration QUERY_SPENT_FACTOR with the default value 0.5, which defines the weight of recent query costs when scheduling. Higher values mean that users with high recent query costs will see their queries penalized harder in the scheduling.

Bug Fixes

  • GraphQL API

    • The SearchDomain.queries GraphQL field has been deprecated. Use SearchDomain.savedQueries instead.

    • The SearchDomain.viewerCanChangeConnections GraphQL field has been deprecated. Use SearchDomain.isActionAllowed instead.

    • The updateSettings GraphQL mutation has been marked as unstable, as it can control unstable and ephemeral settings.

    • Deprecates GraphQL fields UserSettings.isEventListOrderChangedMessageDismissed, UserSettings.isNewRepoHelpDismissed, and UserSettings.settings since they are not used for anything anymore, and will be removed in a future release.

    • Removes the deprecated Repository.isFreemium GraphQL field.

  • Configuration

    • New config: SEGMENTMOVER_EXECUTOR_CORES allows tuning number of concurrent fetches of segments from other nodes to this node. Defaults to vCPU/8, must be at least 2.

    • Removed the QUERY_QUOTA_EXCEEDED_PENALTY configuration.

    • New config: S3_ARCHIVING_IBM_COMPAT for compatility with S3 archiving to IBM Cloud Object Storage.

  • Other

    • When running alerts and scheduled searches, all logging related to a specific alert or scheduled search will now be logged to the humio-activity repository, instead of the humio repository. Error logs will still be logged to the humio repository as well.

    • Fixed a bug which could cause head(), tail(), sort() within either bucket() or a live query to return too few results in certain cases.

    • Fixed a bug where a scheduled search would be executed repeatedly, as long as at least one out of multiple actions was failing. Now, execution is only repeated if all actions are failing.

    • Fixed a bug where cidr() did not respect the include parameter

    • Allow user groups to be represented as a json string and not only as an array when logging in with oauth.

    • Refreshing actions while creating alerts and scheduled searches now happens automatically, but can also be triggered manually using a button.

    • Query poll responses meta data now include Query Quota spent for current user across queries. The cost so far of the current query was there already.

    • Reworked initialization of Humio's async listener infrastructure, to ensure that listeners do not miss any updates. This fixes a number of flakiness issues that could arise when a node was rebooted.

    • Bumped minimum supported versions of Chrome and Chromium from 60 to 69 due to updated dependencies

    • The HEC ingest endpoint is no longer implicitly running kvParse. This used to be the case when ingesting events of the form "event" : "Log line...". If the ingested data is to be key-value parsed, add kvParse() to the relevant parser for the input data.

    • Changed the built-in audit-log parser so that null values are stored as an empty string value. Previously, they were stored as the string "null". The defaults are consistent with the old behavior, so that null values become a "null" string and empty string values are kept.

    • Some minor performance improvements in the ingest pipeline

    • Rework how Humio caches data from global. This fixes a number of data races, where Humio nodes could temporarily get an incorrect view of global.

    • Optimized the splitString() function.

    • Fixed an issue where clicking the label of a parser rerouted erroneously

    • Added documentation link to autocomplete description in the Humio search field

    • Added IP ASN Database license information to the Cluster Administration page

    • Improved error logging for event forwarding

    • Fixed a bug that made it possible to rename a parser to an existing name and thereby overwriting the existing parser.

    • When installing an application package, you sometimes had to refresh the page to get the assets in the package linked to their installed counter parts.

    • Made it possible to delete a parser overriding a built-in parser, even though it is used in an ingest token.

    • Added new parameters handleNull and excludeEmpty to parseJson() to control how null and empty string values are handled.

    • When a query is cancelled, a reason for canceling the query is now always logged. Previously, this was only done if the query was cancelled due to an internal exception. Look for log lines starting with query is cancelled.

    • Fixed a bug that made it impossible to copy a parser to override a built-in parser.

    • Added audit logging when assigning a parser to an ingest token or unassigning a parser from an ingest token. Added the parser name to all audit logs for ingest tokens.

    • Added a new query function: base64Decode()

    • Make the parseLEEF() function more robust and optimize its memory usage.

    • Added a warning to the Cluster Nodes page that warns you if not all Humio servers are running the same Humio version.