Humio Server 1.17.0 Preview (2020-11-18)
Version | Type | Release Date | End of Support | Upgrades From | Data Migration | Config. Changes |
---|---|---|---|---|---|---|
1.17.0 | Preview | 2020-11-18 | 2020-11-26 | 1.16.0 | No | Yes |
JAR Checksum | Value |
---|---|
MD5 | 9c47009810132eb67177966d10471544 |
SHA1 | 341a2607265420b1a2d461ad111c88aca06419ed |
SHA256 | 59001d9a738930f8774c11864ce3c36b3b4b4003f8cad0d4b36af13275ceefa9 |
SHA512 | ee8e2837a8fba2b28aab8d18b16269f452f19a5c80ad9826ecbeb73eaa062236acdf2191cc448e60d81390f1a2881374637adce7603ebf9f0861ca6246f6c82f |
Important Information about Upgrading
Beginning with version 1.17.0, if your current version of Humio is not directly able to upgrade to the new version, you will get an error if you attempt to start up the incompatible version. The 1.17.0 release is only compatible with Humio release 1.16.0 and newer. This means that you will have to ensure that you have upgraded to minimum 1.16.0 before trying to upgrade to 1.17.0. In case you need to do a rollback, this can also ONLY happen back to 1.16.0 or newer, rolling directly back to earlier release can result in data loss.
Humio can now run repeating queries using the
beta:repeating()
function. These are live
queries that are implemented by repeatedly making a query. This
allows using functions in alerts and dashboards that typically
do not work in live queries, such as
selfJoin()
or
selfJoinFilter()
. See the
beta:repeating()
reference page for more
information.
In order to prevent alert notifiers being used to probe services
on the internal network (eg. Zookeeper or the AWS metadata
service), Humio now has an IP filter on alert notifiers. The
default is to block access to all link-local addresses and any
addresses on the internal network; however, you can opt-in to
the old behavior by setting the configuration option
IP_FILTER_NOTIFIERS
to
allow all
. See
IP Filter documentation.
A new experimental query function called
series()
has been added. It needs to be
explicitly enabled on the cluster using the config option
SERIES_ENABLED=true
.
The function series()
improves upon
session()
and
collect()
for grouping events into
transactions. What used to be done with groupby(id,
function=session(function=collect([fields, ...]))) can now be
done using groupby(id, function=series([fields, ..])).
See series()
reference page for more
details.
This new feature stores a copy of live search results to the local disk in the server nodes, and reuses the relevant parts of that cached result when an identical live search is later started. Caching is controlled with the config option QUERY_CACHE_MIN_COST, which has a default value of .0. To disable caching, set the config option to a very high number, such as 9223372036854775807 (max long value).
Improvements, new features and functionality
Functions
New query function parameter to
parseJson()
,removePrefixes
, seeparseJson()
reference page.New query function concatArray, see
concatArray()
reference apge.
Bug Fixes
UI Changes
Setting the default query for a view in the UI has been moved from the "Save as Query" to the View's "Settings" tab.
Automation and Alerts
The notifier list is sorted when selecting notifiers for an alert.
Configuration
New configuration option
ENABLE_ALERTS
makes it possible to disable alerts from running (enabled by default).New configuration option
ALERT_DESPITE_WARNINGS
makes it possible to trigger alerts even when warnings occur.New configuration option
DEFAULT_MAX_NUMBER_OF_GLOBALDATA_DUMPS_TO_KEEP
.New configuration option
IP_FILTER_NOTIFIERS
to set up IP filters for Alert Notifications, see IP Filter reference page.
Functions
New experimental query function
series()
, enabled by configuration optionSERIES_ENABLED=true
, seeseries()
reference page.New query function used to parse events which are formatted according to the Common Event Format (CEF), see
parseCEF()
documentation page.Fixes a bug causing the sub-queries of
join()
etc. to not see events with an @ingesttimestamp occurring later than the search time interval.New experimental query function, see
beta:repeating()
reference page.Fixes a bug causing
join()
to not work after an aggregating function.Fixes a bug where
join()
function in some circumstances would fetch subquery results from other cluster nodes more than once.New experimental query function
window()
, enabled by configuration optionWINDOW_ENABLED=true
, seewindow()
reference page.Fixes a bug causing
sort()
,head()
,tail()
to work incorrectly after other aggregating functions.
Other
Crash the node if an exception occurs while reading from the global Kafka topic, rather than trying to recover.
If
KAFKA_MANAGED_BY_HUMIO
is true, Humio will ensure unclean leader election is disabled on the global-events topic.API Changes (Non-Documented API): Saved Query REST API has been replaced by GraphQL.
Reduce the max fetch size for Kafka requests, as the previous size would sometimes lead to request timeouts.
Fix several cases where Humio might attempt to write a message to Kafka larger than what Kafka will allow.
The
{events_html}
notifier template will now respect the field order from the query.Switch from JDK to BouncyCastle provider for AES decrypt to reduce memory usage.
Fixes an issue where Humio might try to get admin access to Kafka when
KAFKA_MANAGED_BY_HUMIO
was false.Fixes a bug where unit:convert couldn't handle numbers in scientific notation.
Fix negating join expressions.
API Changes (Non-Documented API): View Settings REST API has been replaced by GraphQL.
Reduce the number of merge target updates Humio will write to global on digest leader reassignment or reboot.
Fixes the issue where Humio could behave incompatibly with Kafka versions prior to 2.3.0 if
KAFKA_MANAGED_BY_HUMIO
was true.The Humio-search-all view will no longer be removed if
CREATE_HUMIO_SEARCH_ALL
is set to false. The view will instead become possible to delete manually via the admin UI.Improve logic attempting to ensure other live nodes can act as substitutes in case the preferred digest nodes are not available when writing new segments.
Rename a few scheduler threads so they reflect whether they're associated with streaming queries ("streaming-scheduler") or not ("normal-scheduler")
Periodically release object pools used by mapper pipeline, to avoid a possible source of memory leaks.
Refuse to boot if the global topic in Kafka does not contain the expected starting offset.
Refuse to boot if the booting node would cause violations of the "Minimum previous Humio version" as listed in the release notes.
It is again possible to override a built-in parser in a repository by creating a parser with the same name.
Allow running Humio on JDK-14 and JDK-15 to allow testing these new builds.
Fixes the case where Humio would consider local node state when deciding which ingest data was safe to delete from Kafka.
Tweaked location of diagnostics regarding missing function arguments.
Free-text search has been fixed to behave more in line with the specification.
Fixes an issue causing Humio to fail to upload files to bucket storage in rare cases.
Fixes an issue which caused free-text-search to not work correctly for large (>64KB) events.
Fixes the case where datasources receiving data might not be marked idle, causing Humio to retain too much ingest data in Kafka.
Changed default TLS ciphers and protocols accepted by Humio, see TLS.
Improved wording of diagnostics regarding function arguments.