Humio Server 1.17.0 Preview (2020-11-18)

VersionTypeRelease DateEnd of SupportUpgrades FromData MigrationConfig. Changes
1.17.0Preview2020-11-182020-11-261.16.0NoYes
JAR ChecksumValue
MD59c47009810132eb67177966d10471544
SHA1341a2607265420b1a2d461ad111c88aca06419ed
SHA25659001d9a738930f8774c11864ce3c36b3b4b4003f8cad0d4b36af13275ceefa9
SHA512ee8e2837a8fba2b28aab8d18b16269f452f19a5c80ad9826ecbeb73eaa062236acdf2191cc448e60d81390f1a2881374637adce7603ebf9f0861ca6246f6c82f

Important Information about Upgrading

Beginning with version 1.17.0, if your current version of Humio is not directly able to upgrade to the new version, you will get an error if you attempt to start up the incompatible version. The 1.17.0 release is only compatible with Humio release 1.16.0 and newer. This means that you will have to ensure that you have upgraded to minimum 1.16.0 before trying to upgrade to 1.17.0. In case you need to do a rollback, this can also ONLY happen back to 1.16.0 or newer, rolling directly back to earlier release can result in data loss.

Humio can now run repeating queries using the beta:repeating() function. These are live queries that are implemented by repeatedly making a query. This allows using functions in alerts and dashboards that typically do not work in live queries, such as selfJoin() or selfJoinFilter(). See the beta:repeating() reference page for more information.

In order to prevent alert notifiers being used to probe services on the internal network (eg. Zookeeper or the AWS metadata service), Humio now has an IP filter on alert notifiers. The default is to block access to all link-local addresses and any addresses on the internal network; however, you can opt-in to the old behavior by setting the configuration option IP_FILTER_NOTIFIERS to allow all. See IP Filter documentation.

A new experimental query function called series() has been added. It needs to be explicitly enabled on the cluster using the config option SERIES_ENABLED=true.

The function series() improves upon session() and collect() for grouping events into transactions. What used to be done with groupby(id, function=session(function=collect([fields, ...]))) can now be done using groupby(id, function=series([fields, ..])).

See series() reference page for more details.

This new feature stores a copy of live search results to the local disk in the server nodes, and reuses the relevant parts of that cached result when an identical live search is later started. Caching is controlled with the config option QUERY_CACHE_MIN_COST, which has a default value of .0. To disable caching, set the config option to a very high number, such as 9223372036854775807 (max long value).

Improvements, new features and functionality

Bug Fixes

  • UI Changes

    • Setting the default query for a view in the UI has been moved from the "Save as Query" to the View's "Settings" tab.

  • Automation and Alerts

    • The notifier list is sorted when selecting notifiers for an alert.

  • Configuration

    • New configuration option ENABLE_ALERTS makes it possible to disable alerts from running (enabled by default).

    • New configuration option ALERT_DESPITE_WARNINGS makes it possible to trigger alerts even when warnings occur.

    • New configuration option DEFAULT_MAX_NUMBER_OF_GLOBALDATA_DUMPS_TO_KEEP.

    • New configuration option IP_FILTER_NOTIFIERS to set up IP filters for Alert Notifications, see IP Filter reference page.

  • Functions

    • New experimental query function series(), enabled by configuration option SERIES_ENABLED=true, see series() reference page.

    • New query function used to parse events which are formatted according to the Common Event Format (CEF), see parseCEF() documentation page.

    • Fixes a bug causing the sub-queries of join() etc. to not see events with an @ingesttimestamp occurring later than the search time interval.

    • New experimental query function, see beta:repeating() reference page.

    • Fixes a bug causing join() to not work after an aggregating function.

    • Fixes a bug where join() function in some circumstances would fetch subquery results from other cluster nodes more than once.

    • New experimental query function window(), enabled by configuration option WINDOW_ENABLED=true, see window() reference page.

    • Fixes a bug causing sort(), head(), tail() to work incorrectly after other aggregating functions.

  • Other

    • Crash the node if an exception occurs while reading from the global Kafka topic, rather than trying to recover.

    • If KAFKA_MANAGED_BY_HUMIO is true, Humio will ensure unclean leader election is disabled on the global-events topic.

    • API Changes (Non-Documented API): Saved Query REST API has been replaced by GraphQL.

    • Reduce the max fetch size for Kafka requests, as the previous size would sometimes lead to request timeouts.

    • Fix several cases where Humio might attempt to write a message to Kafka larger than what Kafka will allow.

    • The {events_html} notifier template will now respect the field order from the query.

    • Switch from JDK to BouncyCastle provider for AES decrypt to reduce memory usage.

    • Fixes an issue where Humio might try to get admin access to Kafka when KAFKA_MANAGED_BY_HUMIO was false.

    • Fixes a bug where unit:convert couldn't handle numbers in scientific notation.

    • Fix negating join expressions.

    • API Changes (Non-Documented API): View Settings REST API has been replaced by GraphQL.

    • Reduce the number of merge target updates Humio will write to global on digest leader reassignment or reboot.

    • Fixes the issue where Humio could behave incompatibly with Kafka versions prior to 2.3.0 if KAFKA_MANAGED_BY_HUMIO was true.

    • The Humio-search-all view will no longer be removed if CREATE_HUMIO_SEARCH_ALL is set to false. The view will instead become possible to delete manually via the admin UI.

    • Improve logic attempting to ensure other live nodes can act as substitutes in case the preferred digest nodes are not available when writing new segments.

    • Rename a few scheduler threads so they reflect whether they're associated with streaming queries ("streaming-scheduler") or not ("normal-scheduler")

    • Periodically release object pools used by mapper pipeline, to avoid a possible source of memory leaks.

    • Refuse to boot if the global topic in Kafka does not contain the expected starting offset.

    • Refuse to boot if the booting node would cause violations of the "Minimum previous Humio version" as listed in the release notes.

    • It is again possible to override a built-in parser in a repository by creating a parser with the same name.

    • Allow running Humio on JDK-14 and JDK-15 to allow testing these new builds.

    • Fixes the case where Humio would consider local node state when deciding which ingest data was safe to delete from Kafka.

    • Tweaked location of diagnostics regarding missing function arguments.

    • Free-text search has been fixed to behave more in line with the specification.

    • Fixes an issue causing Humio to fail to upload files to bucket storage in rare cases.

    • Fixes an issue which caused free-text-search to not work correctly for large (>64KB) events.

    • Fixes the case where datasources receiving data might not be marked idle, causing Humio to retain too much ingest data in Kafka.

    • Changed default TLS ciphers and protocols accepted by Humio, see TLS.

    • Improved wording of diagnostics regarding function arguments.