Humio Server 1.31.0 Preview (2021-09-27)
| Version? | Type? | Release Date? | Availability? | End of Support | Security Updates | Upgrades From? | JDK Compatibility? | Req. Data Migration | Config. Changes? |
|---|---|---|---|---|---|---|---|---|---|
| 1.31.0 | Preview | 2021-09-27 | Cloud On-Prem | 2022-10-31 | No | 1.16.0 | 11 | No | Yes |
| JAR Checksum | Value |
|---|---|
| MD5 | 4b4b4411d930d00275bc3a412c0c5c50 |
| SHA1 | ca22652ca303a162f81711f43977e14e528f6e85 |
| SHA256 | 908a907b5c1ffbdce667b932daea0aab46a8ece4e164072f5bae0eb77d3bdd6f |
| SHA512 | bbc65ea8641ac3d9910da4a9dcda0c1817d2a8acc2e81491a389d2671af7e57c12f019ad71af7309d4ad68b67997a6cbd3b0fba3a2048f07b7c7567abd05b8ae |
We now distribute Humio as a tarball in addition to the fat jar format we've previously used. We will continue to distribute the fat jar for the time being. The tarball includes a launcher script, which will set a number of JVM arguments for users automatically. We believe this will help users configure Humio for good performance out of the box. For more information, see Launcher Script.
Search performance via hashfilter-first on segments in buckets
Some searches, including regex and literal string matches, now allow searching without fetching the actual segment files from the bucket, in case the segment is only present in the bucket and not on any local disk. Humio now fetches the hash filter file and uses that to decide if the segment file may have a match before downloading the segment file in this case.
Humio packages can now carry scheduled searches, all types of actions, and files with lookup data (either CSV or JSON formatted). Additionally, we have improved the UI for managing packages, to make it easier to find the package you are looking for. This also marks the point where packages are brought out of beta.
Deprecation
Items that have been deprecated and may be removed in a future release.
Deprecates the copyFile GraphQL mutation, as it is no longer used. If you use this mutation, please let us know.
Improvements, new features and functionality
UI Changes
Updated the style of the email action template and made the wording used dependent on whether an alert or scheduled search was triggered.
The signup path was removed, together with the corresponding pages.
Identity provider pages style update.
The left navigation menu hides, and can be opened again, for mobile devices, on organization settings pages and repository settings pages.
Breadcrumbs are aligned across all pages and show the package name with a link when viewing or editing an asset from a package.
Cluster management pages style updates.
Removed the pop-up link to edit an alert or scheduled search when on the form page. This link is only relevant when creating an entity from the search page via a dialog.
Updated design for Package Marketplace and Installed Packages to make them easier to use and more consistent.
Fixed some styling issue on Query Quotas page.
Automation and Alerts
When selecting actions for alerts or scheduled searches, the actions are now grouped by the package they were imported from.
GraphQL API
Added a GraphQL mutation cancelDeleteEvents that allows cancelling a previously submitted deletion. Cancellation is best-effort, and events that have already been deleted will not be restored.
Deprecates the installPackageFromRegistry and updatePackageFromRegistry GraphQL mutations in favor of installPackageFromRegistryV2 and updatePackageFromRegistryV2.
When using the GraphQL field allowedViewActions, the two previously deprecated actions ChangeAlertsAndNotifiers and ReadEvents are no longer returned. Look for their replacements ChangeTriggersAndActions and ReadContents instead.
Added information about the use of preview fields in the result from calling the GraphQL API. The information will be in the field extensions.preview and will be a list of objects with a name and reason field.
Extended 'Relative' field type for schema files to include support for the value 'now'.
Deprecates the two GraphQL fields id and contentHash on the File type. The two fields are considered unused, so no alternatives are provided. If you rely on them, please let us know.
Deprecates the package field on the SearchDomain GraphQL type, in favor of packageV2. The new field has a simpler and more correct return type.
The name, displayName, and location GraphQL fields on the File type are deprecated in favor of the new nameAndPath field.
The fileName, displayName, and location GraphQL fields on the UploadedFileSnapshot type are deprecated in favor of the new nameAndPath field.
The GraphQL DateTime type now supports non-UTC time. Timestamps like 2021-07-18T14:13:09.517+02.0 are now legal, and will be converted to UTC time internally.
Configuration
The Scheduled Searches feature is no longer in beta and can be used by all users without enabling it first
On a node configured as
USING_EPHEMERAL_DISKS=trueallow the local disk management deleting files even if a query may need them later, as the system is able to re-fetch the files from bucket storage when required. This improves the situation when there are active queries that in total have requested access to more segments than the local disk can hold.Added compatibility mode for using IBM Cloud Object Storage as bucket storage via S3_STORAGE_IBM_COMPAT
Functions
Improved performance in IP database lookups for the functions
ipLocation(),asn()andworldMap().
Other
Fixed an issue with using the browser back button while "advanced editing" the query text of a scheduled search or an alert would hide the blue bar that allows saving the query.
Added support for including dashboard and alert labels when exporting a package.
Warnings when running scheduled searches now show up as errors in the scheduled search overview page if
SCHEDULED_SEARCH_DESPITE_WARNINGSis set to 'false' (the default).Scheduled search "schedule" is explained using human readable text such as "At 9.30 on Tuesdays".
Allow launching using JDK-16.
Improved error handling when running scheduled searches, so that a failed schedules search will be retried as long as it is within the Backfill Limit. [backfill
You can now export and import packages containing any of the action types: Webhook, Email, Humio Repo, Pager Duty, Slack, Slack multi channel, Ops Genie and Victor Ops.
Package installation error messages are now much more readable.
Added focus states to text field, selection and text area components.
The test action functionality no longer uses alert terminology, as actions can be invoked from both alerts and scheduled searches. Also, it is now possible to also test the scheduled search specific message templates using it.
Added Dark Mode for
Query Monitorpage.Improved handling of local disk space relative to
LOCAL_STORAGE_MIN_AGE_DAYS. When the local disk would overflow by respecting that config, Humio can now delete the oldest local segments that are present in bucket storage, even when they are within that time range.Improved search for users page.
Added loading and error states to the page where user selects to create a new repository or view.
Added explicit distribution information for elastic bulk API for elasticsearch API compatibility.
Added support for importing packages with CSV and JSON files. Exporting packages with files is not fully supported yet, but will be in a future release.
Humio docker images is now based on the Alpine linux.
Added maximum width to tabs on the Group page, so they do not keep expanding forever.
Improved audit log for organization creation.
Scheduled search "schedule" field is now validated, showing accurate help for each part of the crontab expression.
Added a Data subprocessors page under account.
Bug Fixes
Documentation
Updated the examples on how to use the
match()query function in the online documentation.
Functions
Other
Fixed an issue where the job responsible for deleting segment files off nodes was not running as often as expected.
Fixed an issue where Shift+Enter would select the current completion rather than adding a newline.
Removed an old Cloud Signups page. The page is not necessary since organizations were implemented for the Cloud environments.
Updated the new asset dialog button text so that it will say 'Continue' when an asset will not be created directly.
When a search is able to filter out segments based on the hash filter files, and a segment file is not present locally on any node, fetch only the hash filter at first, evaluate that, and only if required, fetch the segment file. This speeds up searches that target segments only present in bucket storage and that have search filters that generate hash filter checks, such as regex and literal text comparisons.
Cloning an asset now redirects you to the edit page for the asset for all assets.
Split package export page into dialog with multiple steps.
Amended an internal limit on how many segments can be fetched from bucket storage concurrently. The old limit was based on the number of running queries. The new limit is 32.
Updated dependencies with security fixes.
Fixed an issue where it was possible to submit queries to the Delete Events API that were not valid for that API. Only pure filtering queries are allowed.
Fixed an issue where the query scheduler would spend too much time "shelving" queries, and not enough on getting them executed, leading to little progress on queries.
Fixed an issue where the global consistency check job would fail to perform the consistency check, instead logging lines like "Global dump requested but global had expired". This line can still occur, but only when the consistency check takes too long.
Fixed a bug where a hidden field named "#humioAutoShard" would sometimes show up in the field list.
Fixed an issue that could cause UploadedFileSyncJob to crash if an uploaded file went missing.
Global snapshots are now uploaded to bucket storage more often when there are a lot of updates to it, leading to shorter replay times on startup.
Introduced a check for compatibility for packages and humio versions.
Updated Elastic ingest endpoint to accept 'create' operations in addition to 'index' operations. Both operation types result in the same ingest behavior. This update was added as Fluent-Bit v1.8.3 began using the 'create' operation rather than 'index' for ingest.
Fixed a bug which potentially have caused alerts to not re-fire after the throttle period for field-based throttling had passed.
Fixed an issue where, looking at GraphiQL, the dropdown from the navigation menu was partially hidden.
Truncate long user names on the Users page.
Fixed an issue where the DiskSpaceJob could mark segments accessed slightly out of order during boot.
Fixed thread safety for a variable involved in fetching from bucket storage for queries.
Fixed an issue where Humio attempted to fetch global from other nodes before TLS was initialized.
The simple and advanced permission model has been merged, thus allowing users who were using the simple permission model to create their own permission roles and groups, create groups with default roles, and all other features that were previously only available in advanced permissions mode.
Fixed an issue where new groups added to a repository got a query prefix that disallowed search. The default is now to allow search with the queryprefix *.
The DiskSpaceJob now removes newly written backfilled segments off the local disk before it chooses to remove non-backfilled segments.
Fixed an issue where the job responsible for deleting segment files off nodes was not deleting as many segments as it should.
Fixed an issue where Humio would create a broken hash file for the merge result when merging minisegments that did not originally have hash files.
Updated Slack action for messaging multiple channels, so it propogates errors when triggered. Previously errors were ignored.
Security when viewing installed packages and packages on the marketplace are now less strict. Permissions are still required for installing and uninstalling packages.
Fixed an issue where the {time_zone} Message Templates and Variables for actions would show a full description of the scheduled search instead of only the time zone.
Fixed an issue where certain problems highlighted the first word in a query, not the location of the problem.
Fixed an issue that caused some metrics of type gauge to be reported with a wrong value.
Fixed an issue that caused some errors to be hidden behind a message about "internal error".
The DiskSpaceJob no longer initializes based off of the segment last-modified timestamp, this only happens if no access order snapshot is stored locally. If a snapshot is present, we trust that.
Fixed a bug causing the disk space job to use an expensive code path even when a cheaper one was available.
Fixed an issue where the DiskSpaceJob could continue tracking segments if they were deleted from global, but the files were still present locally.
Reworded a confusing error message when using the
top()function with a limit parameter exceeding the limits configured withTOP_K_MAX_MAP_SIZE_HISTORICALorTOP_K_MAX_MAP_SIZE_LIVE.Fixed an issue that could cause cluster nodes to crash when growing the number of digest partitions.
Creating a new dashboard now opens it after creation.
Fixed an issue where metrics of type gauge with a double value were not reported to the humio-metrics repository, but only to the humio repository.
Fixed a bug where a 404 Not Found status on an internal endpoint would be incorrectly reported as an 401 Unauthorized.
Fixed an issue where Humio would create auxiliary files (hash files) for segments unnecessarily when moving segments between nodes.
When accessing Humio through a URL with either a repository or view name in it and using an ingest token, it is now checked that the view on the token matches the repository or view in the URL, and a 403 Forbidden status is returned, if not.
Fixed a bug on queries that triggered an error while executing due to the input (such as a regex that exceeds limits on execution time) could result in the client getting 404 as status on poll, where it should get .0.