Humio Server 1.40.0 LTS (2022-05-12)

Version?Type?Release Date?Availability?End of Support

Security

Updates

Upgrades

From?

Config.

Changes?
1.40.0LTS2022-05-12

Cloud

2023-05-31No1.30.0Yes

Hide file hashes

Show file hashes

Download: https://repo.humio.com/repository/maven-releases/com/humio/server/1.40.0/server-1.40.0.tar.gz

1.40 REQUIRES minimum version 1.30.0 of Humio to start. Clusters wishing to upgrade from older versions must upgrade to 1.30.0+ first. After running 1.40.0 or later, you cannot run versions prior to 1.30.0.

Behavior Changes

Scripts or environment which make use of these tools should be checked and updated for the new configuration:

  • Configuration

    • The selfJoin() query function was observed to cause memory problems, so we have set a limit of .0.0 output events (there was previously no bound). This limit can be adjusted with the GraphQL mutation setDynamicConfig with configuration flag SelfJoinLimit. A value of -1 returns selfJoin to its old, unbounded version.

New features and improvements

  • Falcon Data Replicator

    • The static configuration variable ENABLE_FDR_POLLING_ON_NODE is no longer supported, as its functionality has been replaced with the dynamic configurations listed above.

    • Introduced dynamic configuration options for changing FDR polling behaviour at runtime. FDR polling is not enabled by default, so you should take care to set up these new configurations after upgrading, or you will risk that your FDR data isn't ingested into Humio before it is deleted from Falcon.

    • Using the dynamic configuration option FdrEnable, administrators can now turn FDR polling on/off on the entire cluster with a single update. Defaults to false.

    • Using the dynamic configuration option FdrMaxNodes, administrators can put a cap on how many nodes should at most simultaneously poll data from the same FDR feed. Defaults to 5 nodes.

    • Using the dynamic configuration option FdrExcludedNodes, administrators can now exclude specific nodes from polling from FDR. Defaults to the empty list, so all nodes will be used for polling.

    • It is now possible to test an FDR feed in the UI, which will test that Humio can connect to the SQS queue and the S3 bucket.

    • Fixed an issue where exceptions in FDR were not properly logged.

  • UI Changes

    • Introducing the new Scatter Chart widget (previously known as XY):

      • It supports long data format (one field for the series name and one field for the y values) as well as wide format (one field per series value).

      • You can now visualize data in the Scatter Chart when queried with the timeChart(), bucket() and groupBy() functions, as well as the table() function like before.

    • Added style options to either truncate or show full legend labels in widgets.

    • Improvements to the Sankey Diagram widget, it now has multiple style options; show/hide the y-axis, sorting type, label position, and colors plus labels for series.

    • Added support in fieldstats() query function for skipping events. This is used by the UI, but only in situations where we know an approximate result is acceptable and where processing all events would be too costly.

    • Improvements to the Pie Chart widget, it now has a max series setting similar to the Time Chart widget.

    • Syntax highlighting for XML, JSON and accesslog data now uses more distinguishable colors.

    • The @timestamp column is now allowed to be moved amongst the other columns in the event list.

    • When using a widget that is not compatible with the current data, the Reset Widget Type button now works again.

    • The widget dropdown can now be navigated with the keyboard.

    • Events with JSON data can now be collapsed and expanded in the JSON panel.

    • Keep empty lines in queries when exporting assets as templates or to packages.

  • GraphQL API

    • Added two new organization level permissions: DeleteAllRepositories and DeleteAllViews that allow repository and view deletion, respectively, inside an organization.

    • The GraphQL queries and mutations for FDR feeds are no longer in preview.

    • Removed the following deprecated GraphQL fields: UserSettings.settings, UserSettings.isEventListOrderChangedMessageDismissed, and UserSettings.isNewRepoHelpDismissed.

    • Changed permission token related GraphQL endpoints to use enumerations instead of strings.

    • It is now possible to refer a parser by name when creating or updating an ingest listener using the GraphQL API mutations createIngestListenerV3 and updateIngestListenerV3. It is now also possible to change the repository on an ingest listener using updateIngestListenerV3. The old mutations createIngestListenerV2 and updateIngestListenerV2 have been deprecated.

    • Removed the deprecated clientMutationId argument from the GraphQL mutation updateSettings.

    • Marked experimental language features as preview in GraphQL API.

    • Added a GraphQL mutation deleteSearchDomainById that deletes views or repositories by ID.

    • It is now possible to refer a parser by name when creating an ingest token or assigning a parser to an existing ingest token using the GraphQL API mutations addIngestTokenV3 and assignParserToIngestTokenV2. The old mutations addIngestTokenV2 and assignParserToIngestToken have been deprecated.

    • Added a new GraphQL mutation to rename views or repositories by ID.

  • Configuration

    • Added a new config NATIVE_FADVICE_SUPPORT (default true) to allow turning off the use of fadvice internally.

    • Amended how Humio chooses segments to download from bucket storage when prefetching. If S3_STORAGE_PREFERRED_COPY_SOURCE is false, the prefetcher will only download segments that are not already on another host. Otherwise, it will download as many hosts as necessary to follow the configured replication factor. This should help avoid excessive bucket downloads when nodes in the cluster have lots of empty disk space.

    • Validate block CRCs before uploading segment files to bucket storage. Can be disabled by setting VALIDATE_BLOCK_CRCS_BEFORE_UPLOAD to false.

    • Added a new config NATIVE_FALLOCATE_SUPPORT (default true) to allow turning off the use of fallocate and ftruncate internally.

    • Require that {S3/GCS}_STORAGE config must be set before {S3/GCS}_STORAGE_2 is set.

    • Added a new configuration variable BUCKET_STORAGE_TRUST_POLICY for the dual-bucket use case. This setting configures which bucket is considered the "trusted" bucket when two buckets are configured, which impacts when Humio considers data to be safely replicated. Supported values are Primary for trusting the primary bucket, Secondary for trusting the secondary bucket, TrustEither for considering data safely replicated if it is in either bucket, and RequireBoth for considering data safely replicated only if it is in both buckets. This config replaces the BUCKET_STORAGE_2_TRUSTED configuration, true in the old configuration equates to Secondary in the new configuration. The default value of the new configuration is Secondary.

  • Dashboards and Widgets

    • Improvements to the Time Chart widget:

      • It now has an option to show the underlying data points, which makes it possible to inspect the behaviour of the different interpolation methods.

      • Trend lines can now be added in the chart.

    • Introducing the Single Value widget. Construct a query which returns any single value, or use the timeChart() query function to create a single-value widget instance with sparkline and trend indicators.

    • Improvements to the Bar Chart widget:

      • Added style options to name the x and y axis.

      • Added option for interpreting the resulting query data as either wide or long format data.

      • Added option to set a max label length for the x-axis, instead of the bottom padding option. With auto-padding and this style option, it is easier to fit the wanted information in the view.

      • It is now possible to configure bar charts to have a logarithmic y axis.

      • Introduced the stacked bar charts option.

      • It no longer has an artificial minimum height for bars, as this may distort at a glance interpretations of the chart.

      • It no longer has sorting by default, which means that the order will be identical to the query result. You can now sort the x axis of the bar chart by using the sort() query function, if sort by series in the style options is not set.

      • It now has a max series setting similar to the Time Chart widget.

  • Functions

    • The findTimestamp() function now supports date formats like 23FEB2022, that is date, literal month and year without any separators in between. Other formats still require separators between the parts.

  • Other

    • Fixed an ingest bug where, under some circumstances, we would reverse the order of events in a batch.

    • Fixed bugs related to repository deletes.

    • It is now possible to create a view with the same name as a deleted view.

    • Fixed an ingest bug where if multiple types of errors occurred for an event we would only add error fields describing one of them. Now we always report all errors.

    • Added a new system-level permission allowing changing the user name of a user.

    • Fixed an issue where OrganizationStatsUpdaterJob would repeatedly post the error com.humio.entities.organization.OrganizationSingleModeNotSupported: Not supported when using organizations in single mode when the cluster was configured for only one organization.

    • Fixed an issue where query cancellation could in rare cases cause the query scheduler to throw exceptions.

    • Fixed how relative time is displayed.

    • Ingest listeners are now only stopped, not deleted, when a user deletes a repository. If the repository is restored, the ingest listener will be restarted automatically. When it is no longer possible to restore the repository, the ingest listener will be deleted.

    • Added support for restoring deleted repositories and views when using bucket storage. See Delete a Repository or View.

    • Humio is now more strict during a Kafka reset to avoid global desyncs. Only one node will be allowed to boot on the new epoch, remaining nodes won't be allowed to use their snapshots, and will need to fetch a fresh global snapshot from that node.

    • If the query scheduler attempts to read a broken segment file, it may be able to fetch a new copy from bucket storage in some cases. Humio will now only allow this if it can be guaranteed that no events from the broken segment have been added to the query result. Otherwise the query will receive a warning.

    • Fixed an ingest bug where we might discard @timezone and @error fields in events with too many fields. Now we always retain those and only discard other fields.

    • Fixed a bug with UTF-8 serialization of 4-byte codepoints (emojis etc.).

    • When Humio detects multiple datasources for the same set of tags, it will not deduplicate them by selecting one source to keep and marking the others replaced.

    • Added humio-token-hashing.sh to the Humio bin directory. This invokes a utility for generating root tokens.

    • Added more visibility on organization limits when changing the retention settings on a repository.

    • Fixed an issue that links in alerts from OpsGenie actions were not clickable.

    • Added humio-decrypt-bucket-file.sh to the Humio bin directory. This invokes a utility for decrypting files downloaded from bucket storage.

    • Fixed an ingest bug where sometimes we wouldn't turn event fields into tags if we fell back to using the key-value parser. Now we always turn fields into tags.

    • It is no longer possible to create ingest listeners on system repositories using the APIs. Previously, it was only prohibited in the UI.

    • Fixed a caching-related issue with groupBy() in live queries that would briefly cause inconsistent results.

    • Webhook action now includes the 'Message Body Template' for PATCH and DELETE requests as well if it is not empty.

    • Fixed a race condition between nodes creating the merge result for the same target segment, and also transferring it among the nodes concurrently. If a query read the file during that race, an in-memory cache of the file header might hold contents that did not match the local file, resulting in Broken segment warnings in queries.

    • Added a feature that allows deletion of repositories and views on cloud.

    • When calculating the starting offset in Kafka for digest, Humio will now trust that if a segment in global is listed as being in bucket storage, that segment is actually present in bucket storage. Humio no longer double checks by asking bucket storage directly.

    • Fixed an issue where download of IOCs from another node in the cluster could start before the previous download had finished, resulting in too many open connections between nodes in the cluster.

    • Fixed an issue where Filebeat 8.1 would not be compatible unless output.elasticsearch.allow_older_versions was set to true.

    • Renamed the Humio tarball distribution to humio-1.39.0.tar.gz instead of humio-release-1.39.0.tar.gz. The file now contains a directory named humio-1.39.0 instead of humio-release-1.39.0.

    • Updating alert labels using the addAlertLabel and removeAlertLabel mutations now requires the ChangeTriggersAndActions permission.

    • Fixed an issue where the UI would not detect parameters in a query when using saved queries from a package.

    • Made changes to Humio's tracking of bucket storage downloads. This should avoid some rare cases where downloads could get stuck.

    • Reduced the amount of time Humio will spend during shutdown waiting for in-progress data to flush to disk to 60 seconds from 150 seconds.

    • Fixed an issue that could cause creation of two datasources for the same tag set if messages with the same tags happened to arrive on different Kafka partitions.

    • During ingest, if an event has too many fields we now sort the fields lexicographically and remove fields from the end. Before, there was no system to which fields were retained, it was effectively random.

    • Adding and removing queries from the query blocklist is now audit logged as two separate audit log event types, query-blocklist-add and query-blocklist-remove, rather than the single event type blocklist.

    • Improved the phrasing of some error messages.

    • Fixed a bug where accessing a csv file with records spanning multiple lines would fail with an exception.

    • The REST API for ingest listeners has been deprecated.

    • Improved distribution of new autosharded datasources.

    • Fixed an issue where an exception in rare cases could cause ingest requests to fail intermittently.

    • The query scheduler improperly handled regex limits being hit, it should result in a warning on the query. In some cases it was handled by retrying the segment read.

    • Fixed an issue where the set-replication-defaults config endpoint could attempt to assign storage to nodes configured not to store segments.

    • Fixed an issue where some errors showed wrong positions in the search page query field.

    • It is no longer possible to delete a parser that is used by an ingest listener. You must first assign another parser to the ingest listener.

    • Fixed an issue where audit logging of alerts, scheduled searches and actions residing on views would yield incomplete or missing audit logs.

    • Fixed an issue where NetFlow parsing would crash if it received an options data record.

    • It is now validated, that the parser supplied when creating or updating an ingest listener, exists.

    • Fixed an ingest bug where, when truncating an event with too many fields, we wouldn't count error fields, leading to the event still being larger than the maximum size.

    • Fixed an issue where Filebeat 8.0 would not be compatible unless setup.ilm.enabled was set to false.

    • Create, update and delete operations on ingest listeners are now always audit logged. Previously, they were only logged when performed through the REST API. Also, the audit log format has been updated to be similar to the format of other assets. Look for events with the type field set to ingestlistener.create, ingestlistener.update, and ingestlistener.delete.

    • Fixed an issue when using bucket storage alongside secondary storage, where Humio would download files to the secondary storage but register them as present in the primary. It will now download and register them as present on the secondary storage.

    • Fixed duplicate Change triggers and actions entry in view permission token page.

    • Fixed an issue that could cause an exception to be thrown in the ingest code if digest assignment changed while a local segment file being written was still empty.

    • Improved performance of formatting action messages, when the query result for an alert or scheduled search contains large events.

    • Improved distribution onto partitions of tag combinations (datasources) that are affected by auto sharding, resulting in less collisions.

    • Improved the flow of creating a blocked query.

    • Humio will now periodically log node configs to the debug log, in addition to the existing log of config on node boot. These logs will come from com.humio.jobs.ConfigLoggerJob.

    • When shared dashboards are disabled or become inaccessible because of IP filters, they will now be completely unreachable, and any dashboards already open will show an informative error message.

    • It is no longer possible to use experimental functions in Alerts, Parsers, and Event Forwarding. They are now only available on the search page.

    • Webhook action has been updated to only allow the following HTTP verbs: GET, HEAD, POST, PUT, PATCH, DELETE and OPTIONS.

    • Added a feature that allows regular users with delete permissions on cloud to rename views and repositories.

    • Fixed an issue where non-default log formats such as log4j2-json-stdout.xml that logs to STDOUT were not fully in control of their output stream, as log entries of level ERROR were also printed directly to stderr from within the code. The default log4j2 configuration now includes a Console appender that prints errors to stdout, achieving the same result, while allowing the other formats to fully control their output stream.

    • Fixed an issue that could cause the query scheduler to erroneously retry searching a bucketed segment.

    • When logging Kafka consumer and producer metrics, Humio will now log repeated metrics like records-lag-max once per partition, with the partition specified in the partition field.

    • Automatic system removals of queries expired from the blocklist are now audit logged as well.