Humio Server 1.51.2 LTS (2022-10-05)

Version?Type?Release Date?Availability?End of Support

Security

Updates

Upgrades

From?

Config.

Changes?
1.51.2LTS2022-10-05

Cloud

2023-08-31No1.30.0No

Hide file hashes

Show file hashes

Download: https://repo.humio.com/repository/maven-releases/com/humio/server/1.51.2/server-1.51.2.tar.gz

These notes include entries from the following previous releases: 1.51.0, 1.51.1

Bug fixes and updates.

Removed

Items that have been removed as of this release.

API

  • The deprecated REST API for actions has been removed, except for the endpoint for testing an action.

  • The deprecated REST API for parsers has been removed.

Deprecation

Items that have been deprecated and may be removed in a future release.

  • Deprecated enabledFeatures query. Use the new featureFlags query instead.

New features and improvements

  • Falcon Data Replicator

    • FDR polling is now turned on by default. Whether FDR polling should be turned on or off on a node can be configured using the ENABLE_FDR_POLLING_ON_NODE configuration variable.

      • If an S3 file is found to be incorrectly formatted during FDR ingest, it will not be ingested completely, but an attempt is made to ingest the remaining S3 files of the SQS message.

      • If an S3 file cannot be found during FDR ingest, it will not be ingested, but an attempt is made to ingest the remaining S3 files of the SQS message.

    • Added environment variable FDR_USE_PROXY which makes the fdr job use the proxy settings specified with: HTTP_PROXY_* environment variables.

  • UI Changes

    • The design of the Time Selector has been updated, and it now features an Apply button on the dashboard page. See Time Interval Settings.

    • Field columns now support multiple formatting options. See Formatting Columns for details.

    • Add missing accessibility features to the login page.

    • In lists of users, with user avatars containing user initials, the current user would sometimes appear to have an opening parenthesis as their last initial.

    • The Live checkbox is now no longer checked automatically when changing the value of the time window in the Time Selector. See Changing Time Interval for details.

    • If Humio fails to start because the cluster is being upgraded, a dedicated message will show when launching the UI.

    • The Save As... button is now always displayed on the Search page, see it described at Saving Searches.

    • Improved keyboard accessibility for creating repositories and views.

    • New styling of errors on search and dashboard pages.

    • Adds an icon and a hint to a disabled side navigation menu item that tells the user the reason for it being disabled.

    • Toggle switches anywhere in the UI can now be accessed using the tab key and can be accessed using the keyboard.

    • When editing an email action in the UI and adding multiple recipients, it is now possible to add a space after the comma in the comma-separated list of recipients.

  • Documentation

    • All documentation links have been updated after the documentation site has been restructured. Please contact support, if you experience any broken links.

  • Automation and Alerts

    • Fixed a bug where an alert with name longer than 50 characters could not be edited.

  • GraphQL API

    • Added preview fields isClusterBeingUpdated and minimumNodeVersion to the GraphQL Cluster object type.

    • Added a new dynamic configuration flag QueryResultRowCountLimit that globally limits how many results (events) a query can return. This flag can be set by administrators through GraphQL. See Limits & Standards for more details.

    • The GQL API mutation updateDashboard has been updated to take a new argument updateFrequency which can currently only be NEVER or REALTIME, which correspond respectively to "dashboard where queries are never updated after first completion" and "dashboard where query results are updated indefinitely".

    • Expose a new GraphQL type with feature flag descriptions and whether they are experimental.

    • Added a GraphQL mutation for testing an action. It is still in preview, but it will replace the equivalent REST endpoint soon.

    • Improved error messaging of GraphQL queries and mutations for alerts, scheduled searches and actions in cases where a given repository or view cannot be found.

  • Configuration

    • Adds a new metric for measuring the merge latency, which is defined as the latency between the last mini-segment being written in a sequence with the same merge target, and those mini-segments being merged. The metric name is segment-merge-latency-ms.

    • Detect need for higher autoshard count by monitoring ingest request flow in the cluster. Dynamically increase the number of autoshards for each datasource to keep flow on each resulting shard below approximately 2MB/s. New dynamic configuration for this that sets the target maximum rate of ingest for each shard of a datasource: TargetMaxRateForDatasource. Default value is 2000000 (2 MB).

    • Added a new environment variable GLOB_MATCH_LIMIT which sets the maximum number of rows for csv_file in match(..., file=csv_file, glob=true) function. Previously MAX_STATE_SIZE was used to determine this limit. The default value of this variable is 20000. If you've changed the value of MAX_STATE_SIZE, we recommend that you also change GLOB_MATCH_LIMIT to the same value for a seamless upgrade.

    • Default value of configuration variable S3_ARCHIVING_WORKERCOUNT raised from 1 to (vCPU/4).

    • Added a new dynamic configuration GroupDefaultLimit. This can be done through GraphQL. See Limits & Standards for details. If you've changed the value of MAX_STATE_LIMIT, we recommend that you also change GroupDefaultLimit and GroupMaxLimit to the same value for a seamless upgrade, see groupBy() for details.

    • Introduced new dynamic configuration LiveQueryMemoryLimit. It can be set using GraphQL. See Limits & Standards for details.

    • Introduced new dynamic configuration JoinRowLimit. It can be set using GraphQL and can be used as an alternative to the environment variable MAX_JOIN_LIMIT. If the JoinRowLimit is set, then its value will be used instead of MAX_JOIN_LIMIT. If it is not set, then MAX_JOIN_LIMIT will be used.

    • Introduced new dynamic configuration StateRowLimit. It can be set using GraphQL. See Limits & Standards for details.

    • Improve the error message if Humio is configured to use bucket storage, but the credentials for the bucket are not configured.

    • Change default value for configuration AUTOSHARDING_MAX from 16 to 128.

    • Add environment variable EULA_URL to specificy url for terms and conditions.

    • Added a link to humio-activity repository for debugging IDP configurations to the page for setting up the same.

    • Bucket storage now has support for a new format for the keys (file names) for the files placed in the bucket. When the new format is applied, the listing of files only happens for the prefixes "tmp/" and "globalsnapshots/". This helps products such a "HCP". The new format is applied only to buckets created after the dynamic configuration BucketStorageKeySchemeVersion has been set to "2". Existing cluster can start using the new format for new files by setting this dynamic configuration. The change will take effect after restarting the cluster. When creating a new Humio cluster, the new format is the default. The new format is supported only on Humio version 1.41+.

    • Introduced new dynamic configuration GroupMaxLimit. It can be set using GraphQL. See Limits & Standards for details.

    • Support for KMS on S3 bucket for Bucket Storage. Specify full ARN of the key. The key_id is persisted in the internal BucketEntity so that a later change of the ID of the key to use for uploads will make Humio still refer the old keyID when downloading files uploaded using the previous key. Setting a new value for the target key results in a fresh internal bucket entity to track which files used kms and which did not. For simplicity it is recommended to not mix KMS and non-KMS configurations on the same S3 bucket.

    • New file format for files uploaded to bucket storage that allows files larger than 2GB to be written to bucket storage. This may be turned on by setting the dynamic configuration BucketStorageWriteVersion to 3. When creating a new Humio clusters, the new format is the default. The new format is supported only on Humio version 1.41+.

    • New configurations BUCKET_STORAGE_SSE_COMPATIBLE that makes bucket storage not verify checksums of raw objects after uploading to an S3. This option is turned on automatically is KMS is enabled (see S3_STORAGE_KMS_KEY_ARN) but is available directly here for use with other S3 compatible providers where verifying even content length does not work.

      Mini segments usually get merged if their event timestamps span more than MAX_HOURS_SEGMENT_OPEN. Mini segments created as part of backfilling did not follow this rule, but will now get merged if their ingest timestamps span more than MAX_HOURS_SEGMENT_OPEN.

    • Adds a new logger job that logs the age of an unmerged miniSegment if the age exceeds the threshold set by the env variable MINI_SEGMENT_MAX_MERGE_DELAY_MS_BEFORE_WARNING. The default value of MINI_SEGMENT_MAX_MERGE_DELAY_MS_BEFORE_WARNING is 2 x MAX_HOURS_SEGMENT_OPEN. MAX_HOURS_SEGMENT_OPEN defaults to 24 hours. The error log produced looks like: Oldest unmerged miniSegment is older than the threshold thresholdMs={value} miniSegmentAgeMs={value} segment={value}.

    • Introduced new dynamic configuration QueryMemoryLimit. It can be set using GraphQL. See also LiveQueryMemoryLimit for live queries. For more details, see Limits & Standards.

  • Dashboards and Widgets

    • Applied stylistic changes for the Inspect Panel used in Widget Editor.

    • Dashboards can now be configured to not update after the initial search has completed. This mode is mainly meant to be used when a dashboard is interactive and not for wall-mounted monitors that should update continually. The feature can be accessed from the Dashboard properties panel when a dashboard is put in edit-mode. See Working in Edit Mode.

    • Bar Chart widget:

      • The Y-axis can now start at smaller values than 1 for logarithmic scales, when the data contain small enough values.

      • It now has an Auto setting for the Input Data Format property, see Wide or Long Input Format for details.

      • Now works with bucket query results.

    • Added empty states for all widget types that will be rendered when there are no results.

    • When importing existing dashboard with a static Shared time, recent changes in the time selection would make those dashboards live.

    • Introducing the Heat Map widget that visualizes aggregated data as a colorised grid.

    • The Pie Chart widget now uses the first column for the series as a fall back option.

    • The Dashboard page now displays the current cluster status.

    • Note widget:

      • Default background color is now Auto.

      • Introduced the text color configuration option.

    • Sorting of Pie Chart widget categories, descending by value. Categories grouped as Others will always be last.

    • The widget legend column width is now based on the custom series title (if specified) instead of the original series name.

    • The Normalize option for the World Map widget has been replaced by a third magnitude mode named None, which results in fixed size and opacity for all marks.

    • Table widgets will now break lines for newline characters in columns.

    • Better handling of dashboard connections issues during restarts and upgrades.

    • Single Value widget:

      • Missing buckets are now shown as gaps on the sparkline.

      • Isolated data points are now visualized as dots on the sparkline.

    • Pie Chart widget now uses the first column for the series as a fall back option.

    • Single Value widget new configuration: deprecated field use-colorised-thresholds in favor of color-method.

      Single Value widget Editor: the configuration option Enable Thresholds is being replaced by an option called Method under the Colors section.

  • Log Collector

    • The Log Collector download page has been enabled for on-prem deployments.

  • Functions

    • Added validation to the field and key parameters of the join() function, so empty lists will be rejected with a meaningful error message.

    • The groupBy() function now accepts max as value for the limit parameter, which sets the limit to the largest allowed value (as configured by the dynamic configuration GroupMaxLimit).

    • Improved the phrasing of the warning shown when groupBy() exceeds the max or default limit.

    • Added validation to the field parameter of the kvParse() function, so empty lists will be rejected with a meaningful error message.

  • Other

    • All users will not have access to the audit log or search all view by default anymore. Access can be granted with permissions.

    • Bump the version of the Monaco code editor.

    • Streaming queries that fail to validate now return a message of why validation failed.

    • Fix a bug causing Humio's digest coordinator to allow nodes to take over digest without catching up to the current leader. This could cause the new leader to replay more data from Kafka than necessary.

    • Fixed an issue where query auto-completion sometimes wouldn't show the documentation for the suggested functions.

    • Adds a new metric for the temp disk usage. The metric name is temp-disk-usage-bytes and denotes how many bytes are used.

    • Added a log message with the maximum state size seen by the live part of live queries.

    • Include the requester in logs from QuerySessions when a live query is restarted or cancelled.

    • The audit log system repository on Cloud has been replaced with a view, so that dashboards etc. can be created on top of audit log data.

    • Make BucketStorageUploadJob only log at info level rather than error if a segment upload fails because the segment has been removed from the host. This can happen if node X tries to upload a segment, but node Y beats it to the punch. Node X may then choose to remove its copy before the upload completes.

    • When unregistering a node from a cluster, return a validation error if it is still alive. Hosts should be shut down before attempting to remove them from the cluster. This validation can be skipped using the same accept-data-loss parameter that also disables other validations for the unregistration endpoint.

    • Added detection and handling of all queries being blocked during Humio upgrades.

    • Added a log of the approximate query result size before transmission to the frontend, captured by the approximateResultBeforeSerialization key.

    • Add flag whether a feature is experimental.

    • Added a log line for when a query exceeds its allotted memory quota.

    • The referrer meta tag for Humio has been changed from no-referrer to same-origin.

    • Compute next set of Prometheus metrics only in a single thread concurrently. If more requests arrive, then the next request gets the previous response.

    • Make a number of improvements to the digest partition coordinator. The coordinator now tries harder to avoid assigning digest to nodes that are not caught up on fetching segments from the other nodes. It also does a better job unassigning digest from dead nodes in edge cases.

    • Fix an unhandled IO exception from TempDirUsageJob. The consequence of the uncaught exception was only noise in the error log.

    • Added a new action type that creates a CSV file from the query result and uploads it to Humio to be used with the match() query function. See Action Type: Upload File.

    • Java in the docker images no longer has the cap_net_bind_service capability and thus Humio cannot bind directly to privileged ports when running as a non-root user.

    • Add warning when a multitenancy user is changing data retention on an unlimited repository.

    • Improved performance of NDJSON format in S3 Archiving.

    • Fix a bug that could cause Humio to spuriously log errors warning about segments not being merged for datasources doing backfilling.

    • Humio now logs digest partition assignments regularly. The logs can be found using the query class=*DigestLeadershipLoggerJob*.

    • All feature flags now contains a textual description about what features are hidden behind the flag.

    • Adds a logger job for cluster management stats it log the stats every 2 minutes, which makes them searchable in Humio.

      The logs belong to the class c.h.c.ClusterManagementStatsLoggerJob, logs for all segments contains globalSegmentStats log about singular segments starts with segmentStats.

    • Remove remains of default groups and roles. The concept was replaced with UserRoles.

Fixed in this release

  • Security

    • Update Netty to address CVE-2022-24823.

    • Bump javax.el to address CVE-2021-28170.

    • Update Scala to address CVE-2022-36944.

  • Falcon Data Replicator

    • FDR Ingest will no longer fail on events that are larger than the maximum allowed event size. Instead, such messages will be truncated.

  • UI Changes

    • Prevent the UI showing errors for smaller connection issues while restarting.

    • Websocket connections are now kept open when transitioning pages, and are used more efficiently for syntax highlighting.

    • Fixed an issue where some warnings would show twice.

    • Intermediate network issues are not reported immediately as an error in the UI.

    • Cloud: Updated the layout for license key page.

    • Fix the dropdown menus closing too early on the home page.

    • Fixed a bug where the "=" and "/=" buttons did not appear on cells in the event list where they should.

    • When viewing the events behind e.g. a Time Chart, the events will now only display with the @timestamp and @rawstring columns.

  • GraphQL API

    • Fix the assets GraphQL query in organizations with views that are not 1-to-1 linked.

  • Configuration

    • Fixed a bug that could result in merging small ("undersized") segments even if the resulting segment would then have a wider than desired time span. The goal it to not produce segments that span more than the 10% of the retention setting for time for the repository. If no time-based retention is configured on the repository, then 3 times the value of configuration variable MAX_HOURS_SEGMENT_OPEN is applied as limit. For default settings, that results in 72 hours.

    • Fixed an issue where event forwarding still showed as beta.

    • Fixed an issue where delete events from a mini-segment could result in the merge of those mini-segments into the resulting target segment never got executed.

      Index in block needs reading from blockwriter before adding each item.

      Fixed a bug where the @id field of events in live query were off by one.

  • Dashboards and Widgets

    • Fixed a bug where certain queries would make it seem that all widgets were incompatible, even though the table view still works.

      Importing a dashboard with Shared time enabled and Live disabled would import the dashboard with Live enabled. Likewise, when creating a new dashboard from a template, Live would be on.

    • The Apply Filter button on the dashboard correctly applies the typed filter again.

    • The theme toggle on a shared dashboard was moved to the header panel and no longer overlaps with any widgets.

    • The Time Chart widget regression line is no longer affected by the interpolation setting.

  • Functions

    • Fixed a bug where using eval as an argument to a function would result in a confusing error message.

    • Fixed a bug where ioc:lookup() would sometimes give incorrect results when negated.

    • Revised some of the error messages and warnings regarding join() and selfJoin().

    • Fixed a recent bug which caused the category links from groupBy()-groups to be lost when a subsequent sort() was used, and also made grouping-based charts (bar, pie, heat map) unusable in such cases.

    • Fixed a bug related to query result metadata for some functions when used as the last aggegate function in a query.

    • Fixed a bug where the writeJson() function would write any field starting with a case-insensitive inf or infinity prefix as a null value in the resulting JSON.

  • Other

    • Make streaming queries search segments newest-to-oldest rather than oldest-to-newest. Streaming queries do not ensure the order of exported events anyway, and searching newest-to-oldest is more efficient.

    • Fix a bug where changing a role for a user under a repository would trigger infinite network requests.

    • Centralise decision for names of files in bucket, allow more than one variant.

      Improved hover messages for strings.

    • Fixed an issue where query auto-completion would sometimes delete existing parentheses.

    • If a segment is deleted or otherwise disappears from global while Humio is attempting to upload it to bucket storage, the upload will now be dropped with an info-level log, rather than requeued with an error log.

    • Fixes the placement of a confirmation dialog when attempting to change retention.

    • Humio will now clean up its tmp directories by deleting all "humiotmp" directories in the data directory when terminating gracefully.

      Fix a regression in the launcher script causing JVM_LOG_DIR to not be evaluated relative to the Humio base install path. All paths in the launcher script should now be relative to the base install path, which is the directory containing the bin folder.

      Fix a bug that could cause merge targets to be cached indefinitely if the associated minis had their mergeTarget unset. The effect was a minor memory leak.

      Fix a bug that could cause Humio to attempt to merge mini-segments from one datasource into a segment in another datasource, causing an error to be thrown.

      When configuring thread priorities, Humio will no longer attempt to call the native setpriority function. It will instead only call the Java API for setting thread priority.

    • Fixed an issue for ephemeral disk based installs where segment files could stay longer on local disks than they were required to, in cases where some nodes listed in the cluster were not alive for extended periods of time.

    • Fixed an issue where JSON parsing on ingest and in the query language was inefficient for large JSON objects.

    • Fix performance issue for users with access to many views.

    • Improve file path handling in DiskSpaceJob to eliminate edge cases where the job might not have been able to tell if a file was on primary or secondary storage.

    • Fix type in Unregisters node text on cluster admin UI.

      • Fixed an issue where event forwarder properties were not properly validated.

      • Reduced the timeout used when testing event forwarders in order to get a better error when timeouts happen.

    • Fix a bug that could cause a NumberFormatException to be thrown from ZooKeeperStatsClient.

    • Fixed an issue where some segments could stall the background process implementing event redaction. This could then result in segments not being merged. The visible symptom would be segments with topOffset attribute being -1, and MiniSegmentMergeLatencyLoggerJob logging that some segments are not being merged.

    • Fix a bug causing digesters to continue digesting even if the local disk is full. The digester will now pause digesting and error log if this occurs.

    • Fix response entities not being discarded in error cases for the proxyqueryjobs endpoint, which could cause warnings in the log.

    • Update org.json:json to address a vulnerability that could cause stack overflows.

    • Fix an issue causing the event forwarding feature to incorrectly reject topic names that contained a dash (-).

    • Fix an issue that could rarely cause exceptions to be thrown from Segments.originalBytesWritten, causing noise in the log.

    • Fix an issue causing Humio to create a large number of temporary directories in the data directory.

    • Bump woodstox to address SNYK-JAVA-COMFASTERXMLWOODSTOX-2928754.

    • Fixed an issue where queries could fail when the requests within the cluster were more than 8 MB each.

    • Some errors messages wrongly pointed to the beginning of the query.

    • Kafka upgrades to 3.2.0 in the Docker images and in the Humio dependencies.

    • Fix a regression introduced in 1.46.0 that can cause Humio to fail to properly replay data from Kafka when a node is restarted.

    • Fixed an issue where strings like Nana and Information could be interpreted as NaN (not-a-number) and infinity, respectively.

    • Fixed a bug where multiline comments weren't always highlighted correctly.