Humio Server 1.32.0 Stable (2021-10-26)

VersionTypeRelease DateAvailabilityEnd of SupportSecurity UpdatesUpgrades FromJDK CompatibilityReq. Data MigrationConfig. Changes
1.32.0Stable2021-10-26Cloud, On-Prem2022-10-31No1.16.011NoYes
JAR ChecksumValue
MD527311d8fb5172cc95aa7cb8d02f5c58c
SHA10378ceb4d0ba9255dba6c213e1a728f7095e8ced
SHA2563dd0434039b1d8eaeb1d1242c860d51593ec222b3d09ae1d732503214edb6d06
SHA512b29267cd2e89cbbf88386a9c2e617c634ff05567388680b605804a0e3978fca4a55c483395d819ff27fbc53da6e29c09ff573c5f7ff00ab57dda45f5e20c1dd2

Download: https://repo.humio.com/repository/maven-releases/com/humio/server/1.32.0/server-1.32.0.tar.gz

We now distribute Humio as a tarball in addition to the fat jar format we've previously used. We will continue to distribute the fat jar for the time being. The tarball includes a launcher script, which will set a number of JVM arguments for users automatically. We believe this will help users configure Humio for good performance out of the box. For more information, see Launcher Script.

Search performance via hashfilter-first on segments in buckets

Some searches, including regex and literal string matches, now allow searching without fetching the actual segment files from the bucket, in case the segment is only present in the bucket and not on any local disk. Humio now fetches the hash filter file and uses that to decide if the segment file may have a match before downloading the segment file in this case.

Humio packages can now carry scheduled searches, all types of actions, and files with lookup data (either CSV or JSON formatted). Additionally, we have improved the UI for managing packages, to make it easier to find the package you are looking for. This also marks the point where packages are brought out of beta.

Deprecation

Items that have been deprecated and may be removed in a future release.

  • Deprecates the two GraphQL fields id and contentHash on the File type. The two fields are considered unused, so no alternatives are provided. If you rely on them, please let us know.

  • Deprecates the copyFile GraphQL mutation, as it is no longer used. If you use this mutation, please let us know.

Improvements, new features and functionality

  • UI Changes

    • Breadcrumbs are aligned across all pages and show the package name with a link when viewing or editing an asset from a package.

    • Updated the style of the email action template and made the wording used dependent on whether an alert or scheduled search was triggered.

    • Updated design for Package Marketplace and Installed Packages to make them easier to use and more consistent.

    • Removed the pop-up link to edit an alert or scheduled search when on the form page. This link is only relevant when creating an entity from the search page via a dialog.

    • Identity provider pages style update.

    • Fixed some styling issue on Query Quotas page.

    • The left navigation menu hides, and can be opened again, for mobile devices, on organization settings pages and repository settings pages.

    • Cluster management pages style updates.

    • The signup path was removed, together with the corresponding pages.

  • GraphQL API

    • Extended 'Relative' field type for schema files to include support for the value 'now'.

    • Deprecates the package field on the SearchDomain GraphQL type, in favor of packageV2. The new field has a simpler and more correct return type.

    • When using the GraphQL field allowedViewActions, the two previously deprecated actions ChangeAlertsAndNotifiers and ReadEvents are no longer returned. Look for their replacements ChangeTriggersAndActions and ReadContents instead.

    • Added a GraphQL mutation cancelDeleteEvents that allows cancelling a previously submitted deletion. Cancellation is best-effort, and events that have already been deleted will not be restored.

    • The GraphQL DateTime type now supports non-UTC time. Timestamps like 2021-07-18T14:13:09.517+02.0 are now legal, and will be converted to UTC time internally.

    • The name, displayName, and location GraphQL fields on the File type are deprecated in favor of the new nameAndPath field.

    • Added information about the use of preview fields in the result from calling the GraphQL API. The information will be in the field extensions.preview and will be a list of objects with a name and reason field.

    • The fileName, displayName, and location GraphQL fields on the UploadedFileSnapshot type are deprecated in favor of the new nameAndPath field.

    • Deprecates the installPackageFromRegistry and updatePackageFromRegistry GraphQL mutations in favor of installPackageFromRegistryV2 and updatePackageFromRegistryV2.

  • Configuration

    • The Scheduled Searches feature is no longer in beta and can be used by all users without enabling it first

    • On a node configured as USING_EPHEMERAL_DISKS=true allow the local disk management deleting files even if a query may need them later, as the system is able to re-fetch the files from bucket storage when required. This improves the situation when there are active queries that in total have requested access to more segments than the local disk can hold.

    • Added compatibility mode for using IBM Cloud Object Storage as bucket storage via S3_STORAGE_IBM_COMPAT

  • Functions

  • Other

    • Limit pending ingest requests by rejecting excess invocations. Rejections are signalled as status 429 "Too many requests" and a Retry-After header suggesting to retry in 5 seconds. Limiting starts when queued requests exceed INGEST_REQUEST_LIMIT_PCT of the total heap size, default is 5.

    • Improved audit log for organization creation.

    • The test action functionality no longer uses alert terminology, as actions can be invoked from both alerts and scheduled searches. Also, it is now possible to also test the scheduled search specific message templates using it.

    • Raise size limit on ingest requests from 8MB to 1 GB

    • Package installation error messages are now much more readable.

    • Allow launching using JDK-16.

    • Added Dark Mode for Query Monitor page.

    • When selecting actions for alerts or scheduled searches, the actions are now grouped by the package they were imported from.

    • Added support for including dashboard and alert labels when exporting a package.

    • Warnings when running scheduled searches now show up as errors in the scheduled search overview page if SCHEDULED_SEARCH_DESPITE_WARNINGS is set to false (the default).

    • Added maximum width to tabs on the Group page, so they do not keep expanding forever.

    • Humio docker images is now based on the Alpine linux.

    • Added a Data subprocessors page under account.

    • Added loading and error states to the page where user selects to create a new repository or view.

    • Improved error handling when running scheduled searches, so that a failed schedules search will be retried as long as it is within the Backfill Limit.

    • You can now export and import packages containing any of the action types: Webhook, Email, Humio Repo, Pager Duty, Slack, Slack multi channel, Ops Genie and Victor Ops.

    • Fixed an issue with using the browser back button while "advanced editing" the query text of a scheduled search or an alert would hide the blue bar that allows saving the query.

    • Added support for importing packages with CSV and JSON files. Exporting packages with files is not fully supported yet, but will be in a future release.

    • Scheduled search "schedule" is explained using human readable text such as "At 9.30 on Tuesdays".

    • Scheduled search "schedule" field is now validated, showing accurate help for each part of the crontab expression.

    • Improved search for users page.

    • Added focus states to text field, selection and text area components.

    • New metric: "ingest-request-delay". Histogram of ingest request time spent being delayed due to exceeding limit on concurrent processing of ingest.

    • Added explicit distribution information for elastic bulk API for elasticsearch API compatibility.

    • Improved handling of local disk space relative to LOCAL_STORAGE_MIN_AGE_DAYS. When the local disk would overflow by respecting that config, Humio can now delete the oldest local segments that are present in bucket storage, even when they are within that time range.

Bug Fixes

  • Documentation

    • Updated the examples on how to use the match() query function in the online documentation.

  • Automation and Alerts

    • Fixed a bug which potentially have caused alerts to not re-fire after the throttle period for field-based throttling had passed.

  • Functions

    • Fixed an issue with the split() function which caused incorrect (usually, too few) query results in some cases where the output fields were refered to later in the query.

    • Fixed an issue where top() with max= can yield the same key multiple times (ei. ...| top([queryId, query], max=totalSize)).

  • Other

    • Fixed an issue where metrics of type gauge with a double value were not reported to the humio-metricsrepository, but only to the humio repository.

    • Fixed an issue where the {time_zone} Message Templates and Variables for actions would show a full description of the scheduled search instead of only the time zone.

    • Removed an old Cloud Signups page. The page is not necessary since organizations were implemented for the Cloud environments.

    • Fixed an issue where Shift+Enter would select the current completion rather than adding a newline.

    • Fixed an issue - when creating a repository a user is automatically assigned a role but isn't able to see himself in the roles list. Also, when editing roles the assignment is not counted correctly under usage.

    • Fixed an issue where Humio attempted to fetch global from other nodes before TLS was initialized.

    • When a search is able to filter out segments based on the hash filter files, and a segment file is not present locally on any node, fetch only the hash filter at first, evaluate that, and only if required, fetch the segment file. This speeds up searches that target segments only present in bucket storage and that have search filters that generate hash filter checks, such as regex and literal text comparisons.

    • Fixed an issue that could cause UploadedFileSyncJob to crash if an uploaded file went missing.

    • Fixed an issue where the global consistency check job would fail to perform the consistency check, instead logging lines like "Global dump requested but global had expired". This line can still occur, but only when the consistency check takes too long.

    • Fixed an issue that caused some metrics of type gauge to be reported with a wrong value.

    • Reworded a confusing error message when using the top() function with a limit parameter exceeding the limits configured with TOP_K_MAX_MAP_SIZE_HISTORICAL or TOP_K_MAX_MAP_SIZE_LIVE.

    • Updated Elastic ingest endpoint to accept 'create' operations in addition to 'index' operations. Both operation types result in the same ingest behavior. This update was added as Fluent-Bit v1.8.3 began using the 'create' operation rather than 'index' for ingest.

    • Fixed an issue that caused some errors to be hidden behind a message about "internal error".

    • Security when viewing installed packages and packages on the marketplace are now less strict. Permissions are still required for installing and uninstalling packages.

    • Fixed an issue where it was possible to submit queries to the Delete Events API that were not valid for that API. Only pure filtering queries are allowed.

    • Updated dependencies with security fixes.

    • Fixed a bug where a 404 Not Found status on an internal endpoint would be incorrectly reported as an 401 Unauthorized.

    • Fixed an issue where Humio would create a broken hash file for the merge result when merging minisegments that did not originally have hash files.

    • Amended an internal limit on how many segments can be fetched from bucket storage concurrently. The old limit was based on the number of running queries. The new limit is 32.

    • Fixed an issue where Humio would retain segments acquired from read-only buckets if those segments were deleted. Humio will now properly delete the segments locally, and drop the reference to the copy in the read-only bucket.

    • Fixed a bug causing the disk space job to use an expensive code path even when a cheaper one was available.

    • Cloning an asset now redirects you to the edit page for the asset for all assets.

    • When accessing Humio through a URL with either a repository or view name in it and using an ingest token, it is now checked that the view on the token matches the repository or view in the URL, and a 403 Forbidden status is returned, if not.

    • Creating a new dashboard now opens it after creation.

    • Global snapshots are now uploaded to bucket storage more often when there are a lot of updates to it, leading to shorter replay times on startup.

    • Truncate long user names on the Users page.

    • Updated Slack action for messaging multiple channels, so it propogates errors when triggered. Previously errors were ignored.

    • Fixed an issue where the query scheduler would spend too much time "shelving" queries, and not enough on getting them executed, leading to little progress on queries.

    • The simple and advanced permission model has been merged, thus allowing users who were using the simple permission model to create their own permission roles and groups, create groups with default roles, and all other features that were previously only available in advanced permissions mode.

    • Fixed thread safety for a variable involved in fetching from bucket storage for queries.

    • Split package export page into dialog with multiple steps.

    • Fixed an issue where the job responsible for deleting segment files off nodes was not deleting as many segments as it should.

    • Fixed an issue where new groups added to a repository got a query prefix that disallowed search. The default is now to allow search with the queryprefix *.

    • Fixed an issue where the DiskSpaceJob could continue tracking segments if they were deleted from global, but the files were still present locally.

    • The DiskSpaceJob now removes newly written backfilled segments off the local disk before it chooses to remove non-backfilled segments.

    • Fixed a bug where a hidden field named "#humioAutoShard" would sometimes show up in the field list.

    • Fixed an issue where certain problems highlighted the first word in a query, not the location of the problem.

    • The DiskSpaceJob no longer initializes based off of the segment last-modified timestamp, this only happens if no access order snapshot is stored locally. If a snapshot is present, we trust that.

    • Introduced a check for compatibility for packages and humio versions.

    • Fixed an issue where the job responsible for deleting segment files off nodes was not running as often as expected.

    • Updated the new asset dialog button text so that it will say 'Continue' when an asset will not be created directly.

    • Fixed an issue where the DiskSpaceJob could mark segments accessed slightly out of order during boot.

    • Fixed an issue where, looking at GraphiQL, the dropdown from the navigation menu was partially hidden.

    • Fixed an issue that could cause cluster nodes to crash when growing the number of digest partitions.

    • Fixed an issue where Humio would create auxiliary files (hash files) for segments unnecessarily when moving segments between nodes.

    • Fixed a bug on queries that triggered an error while executing due to the input (such as a regex that exceeds limits on execution time) could result in the client getting 404 as status on poll, where it should get .0.