Falcon LogScale Documentation

The Time Selector now only allows zooming out to approximately 4,000 years.

The DeleteRepositoryOrView data permission is now visible in the UI on Cloud environments.

A new tutorial built on a dedicated demo data view is available for environments that do not have access to legacy tutorial based on a sandbox repository.

The ChangeRetention data permission is now enabled on Cloud environments.

Regular Expression Syntax new page has been added with extended details of supported regular expression syntax and differences between the LogScale support and other implementations such as Java and Perl.

LogScale Kubernetes Reference Architecture new page has been added with LogScale reference architecture description when deploying LogScale using Kubernetes.

The GraphQL schema for UsageStats has been updated to reflect that queries can be in progress.

The Usage page has been updated to support queries that are in progress for longer than the GraphQL timeout allows.

Setting the SHARED_DASHBOARDS_ENABLED environment variable to false now disables the option of creating links for sharing dashboards.

For more information, see Disabling Access to Shared Dashboards.

Added support for using Google Cloud storage access Workload Identity rather than an explicit service account for bucket storage and export to bucket of query results.

For more information, see Google Cloud Bucket Storage with Workload Identity.

New parsing of Template Expressions has been implemented in the UI for improved performance.

When creating or editing interactions you can now visualize any unused parameter bindings, with the option to remove them.

For more information, see Unused parameters bindings.

The empty list alias is now available as an input option for parameter bindings, so that Multi-value Parameters can be set explicitly to have the value of an empty list.

For more information, see Empty list alias.

Parameter labels are now used instead of parameter IDs when displaying the list of parameters that a widget / query is waiting on.

Polling a query on /queryjobs can now delay the response a bit in order to allow returning a potentially `done` response. The typical effective delay is less than 2 seconds, and the positive effect is saving the extra poll roundtrip that would otherwise need to happen before the query completed. This in particular makes simple queries complete faster from the viewpoint of the client, as they do not have to wait for an extra poll roundtrip in most cases.

When the Kafka broker set changes at runtime, track that set and use as bootstrap servers for Kafka whenever LogScale needs to create a new Kafka client at runtime. This allows replacing all Kafka brokers (incrementally, moving their work to new servers) without restarting LogScale. Note that the set is not persisted across restart of LogScale, so when restarting LogScale, make sure to provide an up to date set of bootstrap servers.

Reduced the amount of memory used when multiple queries use the match() function with the same arguments. Before, if you ran many queries that used the same file, the contents of the file would be represented multiple times in memory, once for each query. This could put you at risk of exhausting the server's memory if the files were large. With this change the file contents will be shared between the queries and represented only once. This enables the server to run more queries and/or handle larger files.

For more information, see Lookup Files Operation.

Fixed an issue where the filter would remain applied in the saved or recent queries when switching tabs in the Queries menu.

Fixed the order of the timezones in the timezone dropdown on the Search and Dashboards pages.

Fixed an issue that could cause some rarely occurring errors when running alerts to not show up on the alert.

Fixed an issue where certain widget options would be ignored when importing a dashboard template or installing a package.

Fixed a wrong behaviour on the Interactions overview page when creating a new interaction: if the interaction panel was opened, the repository options would dropdown in it instead of in the Create new interaction dialog.

The following Node Level Metrics that showed incorrect results are now fixed: primary-disk-usage, secondary-disk-usage, cluster-time-skew, temp-disk-usage-bytes

Removed some code vulnerable to CVE-2023-31442 issue. We do not believe LogScale is affected, as the removed code was experimental and never used in production, but we are removing it to be safe. LogScale does not use the vulnerable component in Akka by default. Users who customize their Akka configuration should ensure they do not set akka.io.dns.resolver = async-dns.

The Alert and Scheduled Search jobs no longer produce logs about specific alerts or scheduled searches in the humio repository. The logs are still sent to the humio-activity repository, which in normal setup is also ingested into the humio repository. So before, the logs would normally be duplicated, now they are not. The only difference between the two types of logs, is that the logs from the humio-activity repository all have loglevel equal to INFO. You can use the severity field instead to distinguish between the severity of the logs.

Mutations enableAlert and disableAlert have been added for enabling and disabling an alert without changing other fields.

Added a new test status for configurations, which allows you to try out a configuration on one or more instances before it’s published.

For more information, see Testing a Remote Configuration.

The following cluster management features are now enabled:

For more information, see Digest Rules.

Degrade and deprecate some REST and GraphQL APIs due to the introduction of AutomaticSegmentDistribution and AutomaticDigesterDistribution. The deprecated elements will be removed in a future release, once the upgrade compatibility with version 1.88.0 is dropped. We expect this to be no earlier than September 2023.

The following REST endpoints are deprecated, as they no longer have an effect and return meaningless results:

The following GraphQL mutations are deprecated, as they no longer have an effect and return meaningless results:

The IngestPartitionScheme mutation is not deprecated, but as it updates state that is overwritten by automation, we recommend against using it — it exists solely to serve as a debugging tool.

The following GraphQL fields on the cluster object are deprecated, and return meaningless values:

The following fields in the return value of the api/v1/clusterconfig/segments/segment-stats endpoint are deprecated and degraded to always be O:

The Search page would reload when using the browser's history navigation buttons. This issue has now been fixed.

An error for lacking permissions that appeared when updating the organization settings has been fixed. Now, if you have permissions to view the Organization Settings page, you can also update information on it.

The throttle field would be empty when editing an Alert; this issue has now been fixed.

Fixed an issue where clicking the Inspect link in Alert notifications would land on a missing page.

The following issues have been fixed on dashboards:

The rename() function would drop the field, if the field and as arguments were identical; this issue has now been fixed.

Fixed an issue that could cause segments to appear missing in queries, due to the presence of deleted minisegments with the same target as live minisegments.

Some merged segments could temporarily be missing from query results right after an ephemeral node reboot. This issue has been fixed.

An edge case has been fixed where query workers could fail to include minisegments if the minisegments were merged at a bad time, causing queries to be missing the data in those segments.

Event List Interactions are now accessible from the Repository Settings page.

Organization level query blocking has been added to Organization Settings UI.

For more information, see Organization Query Monitor.

Improvements in UI tables visualization: even long column headers' text is now always left-aligned (instead of center-aligned and on top of each other) and uses a different color.

The view permission ChangeDashboardReadonlyToken is now also required when creating and deleting shared dashboard tokens.

The error message for an Alert or Scheduled Search on their edit pages now has a button for clearing the error while the dismiss icon will just close the message but not clear errors.

When enabling an Alert or Scheduled Search with no actions, an inline warning message now appears instead of a message box.

The default time window for Alerts has been updated:

For more information, see Creating a New Alert.

When creating a new Alert, you now have a pulldown menu that suggests labels that you’ve previously created for other alerts. The same applies to Scheduled Searches.

For more information, see Creating a New Alert.

Clicking the Labels button in Alerts will now show every unique label that has been created on every alert in the same repository. This means that you don't need to rewrite a label when wanting to add the same label to another alert. This feature also applies to Scheduled Searches.

The following GraphQL mutations can now also be performed with the ChangeOrganizationPermissions permission:

The following GraphQL mutations can now also be performed with the ChangeSystemPermissions permission:

The following GraphQL queries and mutations can now also be performed with either ChangeOrganizationPermissions, ChangeSystemPermissions permission depending on the group:

The permissions required in order to list IP filters have been updated. You can now also list IP filters with one of the following permissions:

The querySearchDomain GraphQL query now allows you to search for Views and Repositories based on your permissions — previously, enforcing specific permissions caused errors.

The new configuration parameter SEGMENT_READ_FADVICE has been introduced.

New configuration parameters have been added allowing control of client.rack for our Kafka consumers:

Automatic rebalancing of existing segments onto cluster nodes has been enabled.

Manual editing of the segment partition table is no longer supported. The table is no longer displayed in the Cluster Administration UI.

The segments will be distributed onto cluster nodes based on the following node-level settings:

Additionally, the following cluster-level setting has been introduced, editable via GraphQL mutations:

This is also configurable via the DEFAULT_SEGMENT_REPLICATION_FACTOR configuration parameter.

If configured via both environment variable and GraphQL mutation, the mutation has precedence.

For new clusters the default is 1. For clusters upgrading from older versions, the initial value is taken from the STORAGE_REPLICATION_FACTOR environment variable, if set. If instead the variable is not set, the value is taken from the replication factor of the storage partition table prior to the upgrade — this means that upgrading clusters should see no change to their replication factor, unless specified in the STORAGE_REPLICATION_FACTOR .

The feature can be disabled in case of problems via either the GraphQL mutation setAllowRebalanceExistingSegments, or the environment variable DEFAULT_ALLOW_REBALANCE_EXISTING_SEGMENTS.

If you need to disable the feature, please reach out to Support and share your concerns so we can try to address them. We intend to remove the option to handle segment partitions manually in the future.

Using the storage class "S3 Intelligent-Tiering" in AWS S3 selectively on files that LogScale knows continues to be supported: it is controlled by the new dynamic configuration BucketStorageUploadInfrequentThresholdDays that sets the minimum number of days of remaining retention for the data in order to switch from the default "S3 Standard" to the "Intelligent" tier.

The decision is made at the point of upload to the bucket only, whereas existing objects in the bucket are not modified.

The bucket must be configured to not allow the optional tiers Archive Access tier nor Deep Archive Access tier as those do not have instant access, which is required for LogScale.

As a consequence of that, do not enable automatic archiving within the S3 Intelligent-Tiering storage class.

Disable the AutomaticDigesterDistribution feature by default. While the feature works, it can cause performance issues on very large installs if nodes are rebooted repeatedly. In future versions, we've worked around this issue, but for 1.88 patch versions, we prefer simply disabling the feature.

Multivalued parameters have been introduced to pass an array of values to the query. The support is limited to the Dashboards page.

For more information, see Multi-value Parameters.

When using the Edit in search view item on a dashboard widget, the values set in parameters in the query are also carried over into the search view.

The new interaction type Update Parameters has been introduced. This interaction allows you to update parameters in the context you’re working in — on the dashboard or on the Search page.

For more information, see Update Parameters.

The combo box has been updated to show multiple selections as "pills".

When Setting Up a Dashboard Interaction, the {{ startTime }} and {{ endTime }} special variables now work differently, depending on whether the query, widget or dashboard is running in Live mode or not. They now work as follows:

Introduced a new setting for dashboard parameters configuration to defer query execution: the dashboard will not execute any queries on page load until the user provides a value to the parameter.

For more information, see Configuring Parameters.

Interactive elements in visualizations now have the point cursor.

The new interaction type Search Link has been introduced, allowing users to create an interaction that will trigger a new search.

For more information, see Managing Dashboard Interactions, Creating Event List Interactions.

You can now save interactions with a saved query on the Search page. Interactions in saved queries are also supported in Packages.

For more information, see Creating Event List Interactions.

You can now delete or duplicate Event List Interactions from the Interactions overview page.

For more information, see Deleting and Duplicating Event List Interactions.

Fleet Management updates:

For more information, see Creating a Configuration - Fleet Management.

On the Config Overview page a column showing the state of the configuration has been added. The configuration can either be published or in draft state.

A menu item has been added on the Config Overview page, that links to the Settings page.

When clicking on an Error status on the Fleet Overview page, a dialog with the error details will open.

For more information, see LogScale Collector Fleet Management.

base64Decode() query function has been updated such that, when decoding to UTF-8, invalid code points are replaced with a placeholder character.

Performance improvements have been made to the match() query function in cases where ignoreCase=true is used together with either mode=cidr, or mode=string.

When IOCs are not available, the ioc:lookup() query function will now produce an error. Previously, it only produced a warning.

The memory usage of the functions selectLast() and groupBy() has been improved.

When the automatic segment rebalancing feature is enabled, ignore the segment storage table when evaluating whether dead ephemeral nodes can be removed automatically.

Reduced the amount of memory used when multiple queries use the match() function with the same arguments. Before, if you ran many queries that used the same file, the contents of the file would be represented multiple times in memory, once for each query. This could put you at risk of exhausting the server's memory if the files were large. With this change the file contents will be shared between the queries and represented only once. This enables the server to run more queries and/or handle larger files.

For more information, see Lookup Files Operation.

Improvements to query scheduler logic for "shelving" i.e., pausing queries considered too expensive. The pause/unpause logic are now more responsive and unpause queries faster when they become eligible to run.

Create Repositories permission now also allows LogScale Self-Hosted users to create repositories.

The size limit of packages' lookup files has been changed to adhere to the MAX_FILEUPLOAD_SIZE configuration parameter. Previously the size limit was 1MB.

For more information, see Exporting the Package.

The Search page would reload when using the browser's history navigation buttons. This issue has now been fixed.

The Fields Panel flyout displayed the bottom 10 values rather than the top 10 values. This issue has now been fixed.

For more information, see Displaying Fields.

An issue in the Usage page that could fail showing any data has been fixed.

The Usage page now shows an error if there are any warnings from the query.

The tooltip in the Time Chart widget would not show any data points. This issue has now been fixed.

'_' was not recognized as a valid first symbol for parameters when parsing queries. This issue has now been fixed.

Non-breaking space chars (ALT+Space) made Template Expressions unable to be resolved. This issue has been fixed.

Attempting to remove a widget on a dashboard would sometimes remove another widget than the one attempted to remove. This issue has been fixed.

"" was being discarded when creating URLs for interactions. This issue has now been fixed.

The values of FixedList Parameter on a dashboard would change sort ordering after being exported to a yaml template file. This issue has been fixed.

Fixed an issue where clicking the Inspect link in alert notifications would land on a missing page.

timeChart() provided with unit and groupBy() as the aggregation function would not warn on exceeding the default groupBy() limit. This issue has now been fixed.

The groupBy() function would not always warn upon exceeding the default limit. This issue has now been fixed.

Fixed a regression in join() validation, which was introduced in version Falcon LogScale 1.80.0 Preview (2023-03-07).

In clusters with bucket storage running queries that take more than 90 minutes, those queries could spuriously fail with a complaint that segments were missing. The issue has now been fixed.

Export query result to file dialog would not close in some cases. This issue has now been fixed.

An issue that would cause bucket downloads to retry infinitely many times for certain types of segments has been fixed.

An uploaded file would sometimes disappear immediately after uploading. This issue has been fixed.

The following audit log issues have been fixed:

An issue that would cause query workers to handle minisegments for longer than intended has been fixed.

Fixed bucket downloads that could fail if the segment they were fetching disappeared from global.

In ephemeral-disk mode, allow removing a node via the UI when it is dead regardless of any data present on the node: ephemeral mode knows how to ensure durability also when nodes are lost without notice.

For more information, see Ephemeral Nodes and Cluster Identity.

Restart of queries based on lookup files has been fixed: only live queries need restarting from changes to uploaded files that they depend on. Scheduled Searches and static queries use the version of the file present when they start and run to completion.

When using the Edit in search view item on a dashboard widget, the values set in parameters in the query are also carried over into the search view.

When Setting Up a Dashboard Interaction, the {{ startTime }} and {{ endTime }} special variables now work differently, depending on whether the query, widget or dashboard is running in Live mode or not. They now work as follows:

base64Decode() query function has been updated such that, when decoding to UTF-8, invalid code points are replaced with a placeholder character.

The memory usage of the functions selectLast() and groupBy() has been improved.

The size limit of packages' lookup files has been changed to adhere to the MAX_FILEUPLOAD_SIZE configuration parameter. Previously the size limit was 1MB.

For more information, see Exporting the Package.

An issue in the Usage page that could fail showing any data has been fixed.

The Usage page now shows an error if there are any warnings from the query.

Non-breaking space chars (ALT+Space) made Template Expressions unable to be resolved. This issue has been fixed.

Attempting to remove a widget on a dashboard would sometimes remove another widget than the one attempted to remove. This issue has been fixed.

timeChart() provided with unit and groupBy() as the aggregation function would not warn on exceeding the default groupBy() limit. This issue has now been fixed.

The groupBy() function would not always warn upon exceeding the default limit. This issue has now been fixed.

In clusters with bucket storage running queries that take more than 90 minutes, those queries could spuriously fail with a complaint that segments were missing. The issue has now been fixed.

When creating a new Alert, you now have a pulldown menu that suggests labels that you’ve previously created for other alerts. The same applies to Scheduled Searches.

For more information, see Creating a New Alert.

New configuration parameters have been added allowing control of client.rack for our Kafka consumers:

'_' was not recognized as a valid first symbol for parameters when parsing queries. This issue has now been fixed.

"" was being discarded when creating URLs for interactions. This issue has now been fixed.

Organization level query blocking has been added to Organization Settings UI.

For more information, see Organization Query Monitor.

Improvements in UI tables visualization: even long column headers' text is now always left-aligned (instead of center-aligned and on top of each other) and uses a different color.

Clicking the Labels button in Alerts will now show every unique label that has been created on every alert in the same repository. This means that you don't need to rewrite a label when wanting to add the same label to another alert. This feature also applies to Scheduled Searches.

The following GraphQL mutations can now also be performed with the ChangeOrganizationPermissions permission:

The following GraphQL mutations can now also be performed with the ChangeSystemPermissions permission:

The following GraphQL queries and mutations can now also be performed with either ChangeOrganizationPermissions, ChangeSystemPermissions permission depending on the group:

The permissions required in order to list IP filters have been updated. You can now also list IP filters with one of the following permissions:

The new configuration parameter SEGMENT_READ_FADVICE has been introduced.

Using the storage class "S3 Intelligent-Tiering" in AWS S3 selectively on files that LogScale knows continues to be supported: it is controlled by the new dynamic configuration BucketStorageUploadInfrequentThresholdDays that sets the minimum number of days of remaining retention for the data in order to switch from the default "S3 Standard" to the "Intelligent" tier.

The decision is made at the point of upload to the bucket only, whereas existing objects in the bucket are not modified.

The bucket must be configured to not allow the optional tiers Archive Access tier nor Deep Archive Access tier as those do not have instant access, which is required for LogScale.

As a consequence of that, do not enable automatic archiving within the S3 Intelligent-Tiering storage class.

Multivalued parameters have been introduced to pass an array of values to the query. The support is limited to the Dashboards page.

For more information, see Multi-value Parameters.

Introduced a new setting for dashboard parameters configuration to defer query execution: the dashboard will not execute any queries on page load until the user provides a value to the parameter.

For more information, see Configuring Parameters.

The new interaction type Search Link has been introduced, allowing users to create an interaction that will trigger a new search.

For more information, see Managing Dashboard Interactions, Creating Event List Interactions.

Fleet Management updates:

For more information, see Creating a Configuration - Fleet Management.

When IOCs are not available, the ioc:lookup() query function will now produce an error. Previously, it only produced a warning.

Worker-level query scheduling has been adjusted to avoid long-term starvation of expensive queries.

Improvements to query scheduler logic for "shelving" i.e., pausing queries considered too expensive. The pause/unpause logic are now more responsive and unpause queries faster when they become eligible to run.

Create Repositories permission now also allows LogScale Self-Hosted users to create repositories.

Fixed a regression in join() validation, which was introduced in version Falcon LogScale 1.80.0 Preview (2023-03-07).

Fixed an issue where a query with join(), selfJoin(), or selfJoinFilter() functions would sometimes get cancelled.

An issue that would cause bucket downloads to retry infinitely many times for certain types of segments has been fixed.

The following audit log issues have been fixed:

An issue that would cause query workers to handle minisegments for longer than intended has been fixed.

Event List Interactions are now accessible from the Repository Settings page.

The default time window for Alerts has been updated:

For more information, see Creating a New Alert.

The querySearchDomain GraphQL query now allows you to search for Views and Repositories based on your permissions — previously, enforcing specific permissions caused errors.

The combo box has been updated to show multiple selections as "pills".

Interactive elements in visualizations now have the point cursor.

You can now save interactions with a saved query on the Search page. Interactions in saved queries are also supported in Packages.

For more information, see Creating Event List Interactions.

You can now delete or duplicate Event List Interactions from the Interactions overview page.

For more information, see Deleting and Duplicating Event List Interactions.

Performance improvements have been made to the match() query function in cases where ignoreCase=true is used together with either mode=cidr, or mode=string.

Fixed bucket downloads that could fail if the segment they were fetching disappeared from global.

Fixed some missing Field Interactions options for the JSON data type in the Event List.

For more information, see Field Data Types.

Improvements have been made on the Fields Panel, that would flicker when switching between the Results and Events tabs and the query was live. It now displays the fields of the aggregated query when on the Results tab, and the fields of the events query when on the Events tab.

The match()query function has been improved in terms of speed when using glob as the mode.

Added backend support for organization level query blocking. Actors with the BlockQueries permission are able to block and stop queries running within their organization.

Worker-level query scheduling has been adjusted to avoid long-term starvation of expensive queries.

Added optional global argument to stopAllQueries, stopStreamingQueries, stopHistoricalQueries, blockedQueries, addToBlocklistById, addToBlocklist permissions. Default is false i.e. within own organization only.

The dropdown menu for dashboard parameter suggestions is now faster and can handle several thousand entries without blocking the UI.

For more information, see Managing Dashboard Parameters.

Fixed an issue where a query with join(), selfJoin(), or selfJoinFilter() functions would sometimes get cancelled.

Fixed an issue that could cause recently merged minisegments to be excluded from searches after a reboot.

Fixed an issue that occurred when creating users: when multiple user creation requests were sent at the same time, multiple users were in some cases created with the same name.

Fixed a permission issue for LogScale Self-Hosted having a dependency on the ManageOrganizations system permission, which should not apply to that environment — the ManageCluster system permission in itself is now sufficient for Self-Hosted.

Fixed an issue with API Explorer that could fail to load in some configurations when using cookie authentication.

The throttle field on alerts can now be imported and exported.

The default value for AUTOSHARDING_TRIGGER_DELAY_MS has been raised from 20,000 to 3,600,000.

Event redaction will no longer rewrite minisegments. Instead, the redaction will be delayed until all minisegments that would be affected have been merged.

Query Monitor page is now available on the organization level. Users with Monitor queries organization level permission get access to the page where they can see queries running in their organization.

For more information, see Query Monitor, Organization Query Monitor.

New ingest endpoint api/v1/ingest/json for ingesting JSON objects and JSON arrays has been added.

For more information, see Ingest Raw JSON Data.

Fixed a bug where testing new FDR feeds that use S3 Aliasing would fail for valid credentials.

The following items have been fixed:

For more information, see Managing Dashboard Interactions.

Fixed a bug where the query editor would wrongly claim that predicate functions used as match guards were missing an argument to the field parameter.

Fixed an issue which could cause minisegments to not all be on the same host for a short time, while those minisegments were being merged. This could cause queries to be unable to query them.

Fixed some issues in the event redaction implementation which could cause the redaction to fail in rare cases.

A bug has been fixed that caused recent minisegments to be missed in queries if the minisegments were merged during the query.

Whether one can create a new repository is now controlled by the Create repository permission in the UI.

Interactions installed from a package use the new repository where the package is installed.

Removed NEW_VHOST_SELECTION_ENABLED as a configuration option. The option has been true by default since 1.70; an opt-out is no longer needed.

Changed the query editor when editing dashboard queries to be the same that is used on the Search page.

New Template feature added to the Fleet Management page, which allows you to:

For more information, see LogScale Collector Fleet Management Overview.

Saved queries can now be used in subqueries, see Using Functions as Arguments to Other Functions.

Added backend support for organization level query monitor. The new MonitorQueries permission now allows viewing queries that are running within the organization.

A high CPU usage in the UI since LogScale 1.75 when the Time Zone Selector dropdown was displayed has now been fixed.

Automatic generation and updating of the digest partitions table has been enabled, and manual editing is no longer supported. See Digest Rules for reference.

The table will be kept up to date based on the following node-level settings (see Assigning Digest Work to Node):

A cluster-level setting has also been introduced: setDigestReplicationFactor GraphQL mutation configures the replication factor to use for the table. This is also settable via the environment variable DEFAULT_DIGEST_REPLICATION_FACTOR.

Automatic management of the digest partition table is now handled by the environment variable DEFAULT_ALLOW_UPDATE_DESIRED_DIGESTERS. We intend to remove the option to handle digest partitions manually in the future.

Keyboard combinations cmd+Z/Ctrl+Z no longer deletes the query on dashboard widgets.

A performance issue in collect() when it collected many values has been fixed.

Validation of join() and join-like functions in conditional expressions and subqueries not having positional information has been fixed.

Fixed an issue where joins in case statements, match statements, and subqueries would mark the entire query as erroneous.

Avoid caching warnings that some data segments could not be found on any servers. This prevents queries from displaying this warning spuriously.

Minisegments would be removed too early from nodes which were querying them, causing queries to be missing some data.

Some minisegments would be excluded from queries in cases where those minisegments had previously been merged, but the merge was reverted.

Two hosts booted at around the same time would conflict on which vhost number to use, causing one of the hosts to crash.

When creating or editing Alerts and Scheduled Searches, it is now possible to specify another user the alert or scheduled search should run as, via the new organization permission ChangeTriggersToRunAsOtherUsers.

It is now checked that the user selected to run the alert or scheduled search has permissions to run it. Previously, that was first checked when trying to run the alert or scheduled search.

The new feature checks whether the user, trying to create or edit an alert or schedule search, has permissions to change and run as another user. If the feature is enabled, you can select the user to run an alert or schedule search as, from a list of users.

See Creating a New Alert and Scheduled Search Permissions for more information.

Introduced a memory limit in collect() mapper phase. The collect() function now collects up to the value of the limit argument or 10 MiB worth of distinct values, whichever comes first.

Memory consumption of the format() function has been decreased.

Fixed a performance issue when setting fileDownloadParallelism to more than 1. See Adjust Polling Nodes Per Feed for more information.

The Event Distribution Histogram wouldn't show properly after manipulation of the @timestamp field.

In visualizations using the timeChart() or bucket() functions, when no results were returned you would just see an empty page. Consistently with other visualizations, you will now see a no-result message displayed, such as No results in active time window or Search Completed. No results found — depending on whether Live mode is selected or not.

Fixed dashboard links to the same dashboard, as they would not correctly update the parameters.

The Time Zone Selector now shows the timezone as +00:00 instead of -00:00 when the offset is zero.

Clone items have all been replaced with Duplicate in the UI to be consistent with what they actually do.

An explicit logout message now indicates that the user's session has been terminated.

Removed the sidepanel when creating/editing an Alerts or Scheduled Searches.

When updating or creating new Actions, any server errors will be displayed in a summary under the form. The server errors in the summary will now specify the form field title where the error occurred, to easily identify where the error is.

The default value of MAX_INGEST_REQUEST_SIZE has been reduced from 1024 MB to 32 MB. This limits the size of ingest requests and rejects oversized requests. If the request is compressed within HTTP, then this restricts the size after decompressing.

Introduced the new query function bitfield:extractFlags().

More time format modifiers are now supported in the format() function:

The array:filter() function is now generally available.

"Sticky" autoshards no longer mean that the system cannot tune their value, but only that it cannot decrease the number of shards; the cluster is allowed to raise the number of shards on datasources when it needs to, also for those that were set as sticky using the REST API.

An enhancement has been made so that when the number of ingest partitions is increased, fresh partitions are assigned to all non-idle datasources based on the new set of partitions. Before this change only new datasources (new tag combinations) would be using the new partitions. The auto-balancing does not start if there are nodes in the cluster running versions prior to 1.78.0.

When exporting a dashboard, alert or scheduled search as a template, the labels' field was missing in the exported YAML.

For more information, see Managing Alerts, Scheduled Searches, Using Dashboards.

Double-clicking in the Event List would open the Inspection Panel instead of making a text selection. It now correctly selects the word being double-clicked.

A typo has been fixed in message ActionWithIdNotFound.

Pending deletes that would cause nodes to fail to start, reporting a NullPointerException, have been fixed.

A newly added, unconfigured dashboard parameter could not be deleted again. This issue has been fixed.

Fixed ingest-only nodes that would fail all requests to /dataspaces and /repositories.

Prevent nodes configured not to run queries from starting queries locally in the case where the query request can't be proxied.

When making updates to the query partition table, only change partitions with dead nodes. This should allow queries to continue without requiring resubmit when a previously unknown node joins the cluster.

Ensure we keep hosts listed in the query partition table up to date as those hosts restart. This should prevent an issue where removing too many nodes from a cluster could prevent queries from running.

Filtering and group-by icons have been added to the Fields Panel and Inspection Panel detail views.

The new Template Language reference information section has been added.

Hold ⇧ (Shift) to show unformatted values. Hold ⌥ (Alt on Windows or Option on Mac) to show full legend labels.

startTime, endTime, and parameter variables are now also available when working with Template Language expressions on the Search page.

Introduced the array:reduceAll() function.

As other released Array Query Functions, it requires square braces.

Adding more Repositories and Views to a group is now done inside a dialog.

Ephemeral nodes are automatically removed from the cluster if they are offline for too long (2 hours by default).

Repository interactions are now supported in Packages. When exporting a package with dashboard link interactions referencing a dashboard also included in the package, then that reference will be updated to reflect this in the resulting zip file.

When using the Export as template functionality, the label field was missing in the exported YAML.

For more information, see Using Dashboards.

The Scatter Chart widget visualization would under some conditions claim to be compatible with any result that has 3 or more fields. Yet it would not display anything unless the actual data was numeric. The Scatter Chart visualization now properly detects compatibility and ignores any non-numeric fields in the query result.

The Table widget is able to display any search result, yet in the widget dropdown, it would often say "Incompatible". It now indicates compatibility. For event type results, the Event List visualisation will still be preferred and auto selected.

When importing a dashboard from a template, some widget options (including LegendPosition) were being ignored and reverted to their default value.

If you clone a widget and click Edit in Search View, you would be asked to discard your changes before editing, causing confusion. Now, Edit in Search View is not available until you save or discard using the buttons in the top bar.

For more information, see Managing Dashboard Widgets, Editing an Existing Widget.

The collect() function has been fixed in that its limit parameter was not being obeyed. This would lead to inconsistent results when there were more values to collect than what specified in the limit.

Timeout from publish to global topic in Kafka has been fixed, as it resulted in marking input segments for merge as broken temporarily.

Nodes are now considered ephemeral only if they set USING_EPHEMERAL_DISKS to true. Previously, they were ephemeral if they either set that configuration, or if they were using the httponly node role.

Linked to the correct SaaS eula for SaaS customers.

Fixing minisegment downloads during queries, as they could cause download retries to fail spuriously, even if the download actually succeeded.

Job-to-node assignment in Logscale has been reworked. Jobs that only needed to run on a subset of nodes in the cluster — such as the job for firing alert notifications or the job enforcing retention settings — would previously select which hosts were responsible for executing the job based on the segment storage table.

The selection is now based on consistent hashing, which means the job assignments should automatically follow the set of live nodes.

It is possible to observe where a given job is running based on logs found with the query class=*JobAssignments*.

Fixing minisegment fetches as they failed to complete properly during queries, if the number of minisegments involved was too large.

Fixed an issue with API Explorer that could fail to load in some configurations when using cookie authentication.

Removed the API for managing ingest tokens. This has long been deprecated and replaced by a GraphQL API.

The Search page and Dashboards URLs support links with timezones, to sharing links ensure the same timezone. E.g. ?tz=Europe/Copenhagen

Known field names are now shown as completion suggestions in The Search Box while you type.

Suggestions in The Search Box will show for certains function parameters like time formats.

Changes have been made for the three-dot menu (⋮) used for Field Interactions:

Tabs on the Users page are renamed: former Groups and Permissions tab is now renamed to Permissions; former Details tab is now renamed to Information. In addition, the Permissions tab is now displayed first — it is also the tab that will be opened by default when navigating to a user from other places in the product. See Role Based Access Control (RBAC) for a description of roles and permissions in the UI.

Event List Interactions are now sorted by name and repository name by default.

You can now set your preferred timezone under Manage your Account.

The Search page now supports timezone picking. The timezone will be set on the users' session and remembered between pages.

Introduced Search Interactions to add custom event list options for all users in a repository.

For more information, see Event List Interactions.

The list of Message Templates and Variables is no longer shown in the User Interface when editing Actions, instead a link to the documentation has been added.

GraphQL API mutations have been added for testing actions without having to save them first. The added mutations are:

The previous testAction mutation has been removed.

The new GraphQL API mutations' signature is almost the same as the create mutation for the same action, except that test actions require event data and a trigger name, as the previous testAction mutation did.

As a consequence, the Test button is now always enabled in the UI.

New dynamic configuration FlushSegmentsAndGlobalOnShutdown. When set, and when USING_EPHEMERAL_DISKS is set to true, forces all in-progress segments to be closed and uploaded to bucket, and also forces a write (and upload) of global snapshot during shutdown. When not set, this avoids the extra work and thus time shutting down from flushing very recent segments, as those can then be resumed on next boot, assuming that next boot continues on the same Kafka epoch. The default is false, which allows faster shutdown.

A new environment configuration variable GLOB_ALLOW_LIST_EMAIL_ACTIONS is introduced. It enables cluster-wide blocking of recipients of Email actions that are not in the provided allow list.

The ability to keep the same merge target across digest changes is reintroduced. This feature was reverted in an earlier release due to a discovered issue where mini segments for an active merge target could end up spread across hosts. As that issue has been fixed, mini segments should now be stored on the hosts running digest for the target.

The Single Value widget now supports interactions on both the Search and Dashboard page. See Managing Dashboard Interactions for more details on interactions.

It is now possible to set a temporary timezone in dashboards, which will be read from the URL on page load.

For more information, see Time Interval Settings.

Introduced Dashboards Interactions to add interactive elements to your dashboards.

For more information, see Managing Dashboard Interactions.

LogScale Collector Fleet Management now supports remote configuration of LogScale Collectors. This gives an administrator the option of managing the configuration of LogScale Collector instances in LogScale, instead of managing configuration files directly on the device where Falcon LogScale Collector is installed.

For more information, see LogScale Collector Fleet Management, LogScale Collector Releases.

The query function holtwinters() has been removed from the product.

Using ioc:lookup() in a query while the IOC service is disabled will now result in a failed query instead of a warning, stating that there are partial results.

Queries containing a join() function no longer run truly live when the query is set to Live. Instead, these queries will run repeatedly at intervals determined by the query engine.

For more information, see Warnings when Using Live join() Functions, Creating Widgets with Live join() Functions, join(), Special Behaviour for Live Joins.

The performance of in() is improved when matching with values that do not use the * wildcard.

default() now supports assigning the same value to multiple fields, by passing multiple field names to the field parameter.

selectLast() and groupBy() now use less state size, allowing for larger result sets.

"Sticky" autoshards no longer mean that the system cannot tune their value, but only that it cannot decrease the number of shards; the cluster is allowed to raise the number of shards on datasources when it needs to, also for those that were set as sticky using the REST API.

Ephemeral nodes are automatically removed from the cluster if they are offline for too long (2 hours by default).

When creating a new group you now have to add the group and add permissions for it in the same multi step dialog.

Fixed an issue where the dashboard page would freeze when the value of a dashboard parameter was changed.

Fixed the UI as it were not showing an error when a query gets blocked due to query quota settings.

Fixed an issue that made switching UI theme report an error and only take effect for the current session.

We have fixed tooltips in the query editor, which were hidden by other elements in the UI.

For self-hosted: Postmark for sending emails from Actions no longer uses the IP filter, allowing administrators not to put Postmark on the IP allowlist.

Pending deletes that would cause nodes to fail to start, reporting a NullPointerException, have been fixed.

Removed compression type extreme for configuration COMPRESSION_TYPE. Specifying extreme will now select the default value of high in order not to cause configuration errors for clusters that specify extreme. The suggestion is to remove COMPRESSION_TYPE from your configurations unless you specify the only other non-default value of fast.

Fixed an issue where the IOC database could get out of sync. The IOC database will be re-downloaded upon upgrade, therefore IOCs won’t be completely available for a while after the upgrade.

Queries ending with tail() will no longer be rendered with infinite scroll.

We have reduced the noise from MiniSegmentMergeLatencyLoggerJob by being more conservative about when we log mini segments that are unexpectedly not being merged. We have made MiniSegmentMergeLatencyLoggerJob take datasource idleness into account.

Timeout from publish to global topic in Kafka has been fixed, as it resulted in marking input segments for merge as broken temporarily.

Nodes are now considered ephemeral only if they set USING_EPHEMERAL_DISKS to true. Previously, they were ephemeral if they either set that configuration, or if they were using the httponly node role.

Fixing minisegment downloads during queries, as they could cause download retries to fail spuriously, even if the download actually succeeded.

Fixed a failing require from MiniSegmentsAsTargetSegmentReader, causing queries to fail in very rare cases.

Fixed an issue where the query scheduling could hit races with the background recompression of files in a way that resulted in the query missing the file and ending up adding warnings about segment files being missed by the query.

Fixing minisegment fetches as they failed to complete properly during queries, if the number of minisegments involved was too large.

Fixed an issue for the ingest API that made it possible to ingest into system repositories.

We have set a maximum number of events that we will parse under a single timeout so large batches are allowed to take longer. If you've seen parsers time out not because the parser is actually slow but because you were processing many events in a single batch, this change should cause that stop happening. Only parsers that are genuinely slow should now time out.

Removed the API for managing ingest tokens. This has long been deprecated and replaced by a GraphQL API.

Event List Interactions are now sorted by name and repository name by default.

The Single Value widget now supports interactions on both the Search and Dashboard page. See Managing Dashboard Interactions for more details on interactions.

It is now possible to set a temporary timezone in dashboards, which will be read from the URL on page load.

For more information, see Time Interval Settings.

LogScale Collector Fleet Management now supports remote configuration of LogScale Collectors. This gives an administrator the option of managing the configuration of LogScale Collector instances in LogScale, instead of managing configuration files directly on the device where Falcon LogScale Collector is installed.

For more information, see LogScale Collector Fleet Management, LogScale Collector Releases.

Queries containing a join() function no longer run truly live when the query is set to Live. Instead, these queries will run repeatedly at intervals determined by the query engine.

For more information, see Warnings when Using Live join() Functions, Creating Widgets with Live join() Functions, join(), Special Behaviour for Live Joins.

Fixed an issue for the ingest API that made it possible to ingest into system repositories.

The Search page and Dashboards URLs support links with timezones, to sharing links ensure the same timezone. E.g. ?tz=Europe/Copenhagen

Known field names are now shown as completion suggestions in The Search Box while you type.

Suggestions in The Search Box will show for certains function parameters like time formats.

You can now set your preferred timezone under Manage your Account.

The Search page now supports timezone picking. The timezone will be set on the users' session and remembered between pages.

Introduced Search Interactions to add custom event list options for all users in a repository.

For more information, see Event List Interactions.

GraphQL API mutations have been added for testing actions without having to save them first. The added mutations are:

The previous testAction mutation has been removed.

The new GraphQL API mutations' signature is almost the same as the create mutation for the same action, except that test actions require event data and a trigger name, as the previous testAction mutation did.

As a consequence, the Test button is now always enabled in the UI.

Introduced Dashboards Interactions to add interactive elements to your dashboards.

For more information, see Managing Dashboard Interactions.

The performance of in() is improved when matching with values that do not use the * wildcard.

default() now supports assigning the same value to multiple fields, by passing multiple field names to the field parameter.

selectLast() and groupBy() now use less state size, allowing for larger result sets.

Fixed the UI as it were not showing an error when a query gets blocked due to query quota settings.

Fixed an issue that made switching UI theme report an error and only take effect for the current session.

For self-hosted: Postmark for sending emails from Actions no longer uses the IP filter, allowing administrators not to put Postmark on the IP allowlist.

Queries ending with tail() will no longer be rendered with infinite scroll.

Fixed a failing require from MiniSegmentsAsTargetSegmentReader, causing queries to fail in very rare cases.

Unlimited waits for nodes to get in sync has been fixed. This caused digest coordination to fail, to limit the time allowed for a node to get "in sync" on a partition before leadership was assigned to it, in cases where the previous digest leader shut down gracefully.

Changes have been made for the three-dot menu (⋮) used for Field Interactions:

The list of Message Templates and Variables is no longer shown in the User Interface when editing Actions, instead a link to the documentation has been added.

A new environment configuration variable GLOB_ALLOW_LIST_EMAIL_ACTIONS is introduced. It enables cluster-wide blocking of recipients of Email actions that are not in the provided allow list.

When creating a new group you now have to add the group and add permissions for it in the same multi step dialog.

Fixed an issue where the dashboard page would freeze when the value of a dashboard parameter was changed.

We have fixed tooltips in the query editor, which were hidden by other elements in the UI.

Fixed an issue where the environment variable OIDC_USE_HTTP_PROXY was not respected. It means that LogScale will now call all OIDC endpoints directly without going through the HTTP Proxy, when OIDC_USE_HTTP_PROXY is set to false.

This fixes the known issue previously reported in Falcon LogScale 1.63.1 Stable (2022-11-14), Falcon LogScale 1.63.2 Stable (2022-11-30), Falcon LogScale 1.63.3 Stable (2022-12-21) and Falcon LogScale 1.70.0 Stable (2023-01-16).

Some restrictions have been introduced when running with single-user as AUTHENTICATION_METHOD:

We have set a maximum number of events that we will parse under a single timeout so large batches are allowed to take longer. If you've seen parsers time out not because the parser is actually slow but because you were processing many events in a single batch, this change should cause that stop happening. Only parsers that are genuinely slow should now time out.

The query function holtwinters() has been removed from the product.

Using ioc:lookup() in a query while the IOC service is disabled will now result in a failed query instead of a warning, stating that there are partial results.

We have reduced the noise from MiniSegmentMergeLatencyLoggerJob by being more conservative about when we log mini segments that are unexpectedly not being merged. We have made MiniSegmentMergeLatencyLoggerJob take datasource idleness into account.

Removed compression type extreme for configuration COMPRESSION_TYPE. Specifying extreme will now select the default value of high in order not to cause configuration errors for clusters that specify extreme. The suggestion is to remove COMPRESSION_TYPE from your configurations unless you specify the only other non-default value of fast.

Fixed an issue where the query scheduling could hit races with the background recompression of files in a way that resulted in the query missing the file and ending up adding warnings about segment files being missed by the query.

Tabs on the Users page are renamed: former Groups and Permissions tab is now renamed to Permissions; former Details tab is now renamed to Information. In addition, the Permissions tab is now displayed first — it is also the tab that will be opened by default when navigating to a user from other places in the product. See Role Based Access Control (RBAC) for a description of roles and permissions in the UI.

New dynamic configuration FlushSegmentsAndGlobalOnShutdown. When set, and when USING_EPHEMERAL_DISKS is set to true, forces all in-progress segments to be closed and uploaded to bucket, and also forces a write (and upload) of global snapshot during shutdown. When not set, this avoids the extra work and thus time shutting down from flushing very recent segments, as those can then be resumed on next boot, assuming that next boot continues on the same Kafka epoch. The default is false, which allows faster shutdown.

The ability to keep the same merge target across digest changes is reintroduced. This feature was reverted in an earlier release due to a discovered issue where mini segments for an active merge target could end up spread across hosts. As that issue has been fixed, mini segments should now be stored on the hosts running digest for the target.

Fixed an issue where the IOC database could get out of sync. The IOC database will be re-downloaded upon upgrade, therefore IOCs won’t be completely available for a while after the upgrade.

Ephemeral nodes are automatically removed from the cluster if they are offline for too long (2 hours by default).

Pending deletes that would cause nodes to fail to start, reporting a NullPointerException, have been fixed.

Timeout from publish to global topic in Kafka has been fixed, as it resulted in marking input segments for merge as broken temporarily.

Nodes are now considered ephemeral only if they set USING_EPHEMERAL_DISKS to true. Previously, they were ephemeral if they either set that configuration, or if they were using the httponly node role.

Fixed an issue where the environment variable OIDC_USE_HTTP_PROXY was not respected. It means that LogScale will now call all OIDC endpoints directly without going through the HTTP Proxy, when OIDC_USE_HTTP_PROXY is set to false.

This fixes the known issue previously reported in Falcon LogScale 1.63.1 Stable (2022-11-14), Falcon LogScale 1.63.2 Stable (2022-11-30), Falcon LogScale 1.63.3 Stable (2022-12-21) and Falcon LogScale 1.70.0 Stable (2023-01-16).

Some restrictions have been introduced when running with single-user as AUTHENTICATION_METHOD:

Unlimited waits for nodes to get in sync has been fixed. This caused digest coordination to fail, to limit the time allowed for a node to get "in sync" on a partition before leadership was assigned to it, in cases where the previous digest leader shut down gracefully.

Added support for export and import of dashboards with query based widgets which use a fixed time window.

New background task TagGroupingSuggestionsJob that reports on flow rate in repositories with many datasources on what it considers slow ones, controlled by configuration of segment sizes and flush intervals. The output in the log can be input to decision on add Tag Grouping to a repository to reduce the number of slow datasources.

Add code to ensure all minisegments for the same target end up located on the same hosts. A change in 1.63 could create a situation where minisegments for the same merge target wound up on different nodes, which the query code currently assumes can't happen. This could cause Result is partial responses to user queries.

Fixed a bug where a link in the notification for a failed alert would link to a non-existing page.

Fixed an issue with parameters in dashboards, where the values of a fixed list parameter would not have their order maintained when exporting and importing templates.

Scatter Chart has been updated:

Fixed three bugs in the Bar Chart — where the sorting would be wrong with updating query results in the stacked version, flickering would occur when deselecting all series in the legend, and deselecting renamed series in the legend would not have any effect.

Fixed a bug where very long string literals in a regex could cause a query/parser to fail with a stack overflow.

OIDC known issue preventing login:

if you are running LogScale self-hosted, this version will not work in an environment where you are Authenticating with OpenID Connect and you use an HTTP Proxy, but you do not want to call OpenIDConnect through the HTTP Proxy.

Please do not upgrade to this version if this is the case — upgrade to Falcon LogScale 1.63.4 Stable (2023-02-01) and Falcon LogScale 1.70.1 Stable (2023-02-01) instead, where this issue has been fixed.

Add bounds to maximum number of active notifications per user.

Throttle publish to global-events topic internally based on time spent in recent transactions of the same type (digest-related writes are not throttled). See details at new configuration variable GLOBAL_THROTTLE_PERCENTAGE page. Also see the metric global-operation-time for measurements of the time spent.

Added option to filter by group and role permission types in groupsPage and rolesPage queries.

Reduced CPU usage of background tasks for the case of high partition count and high datasource count.

Add support for GET+DELETE requests for queries by external Query ID without including the repository name in the URL. The new URL is /api/v1/queryjobs/QUERYID. Note that shared dashboard token authentication is not supported on this API. (The existing API on /api/v1/repositories/REPONAM/queryjobs/QUERYID remains unmodified and support POST requests for submit of queries.)

LogScale no longer considers every host to be alive for a period after rebooting. Only hosts marked as running in global will be considered alive. This fixes an issue where a query coordinator might pointlessly direct queries to dead nodes because the coordinator had recently booted.

Allow creating Kafka topics even if a broker is down.

The query function holtwinters() is now deprecated and will be removed along with the release of future version 1.73; therefore, its usage in alerts is not recommended.

Enforcing S3 file size limits (30MB) in FDR feeds. Files will not be ingested if they are above the limit.

Introduced the Social Login Settings feature: all customers with access to the organization identity providers page can now change social login settings in the UI. See Configuring Identity Providers for LogScale Cloud for details.

No longer possible to add a color to roles. Existing role colors removed from the UI.

Improved performance when storing alert errors and trigger times in global.

Set dynamic configuration BucketStorageWriteVersion to 3. This sets the format for files written to bucket storage to use a format that allows files larger than 2GB and incurs less memory pressure when decrypting files during download from the bucket. The new format is supported since 1.44.0 only.

Changed the default value for AUTHENTICATION_METHOD from none to single-user.

To set username and password, use the environment variables:

See SINGLE_USER_USERNAME and SINGLE_USER_PASSWORD documentation for more details on these variables.

Set minimum version for the cluster to be 1.44.0.

New metric global-operation-time: tracks local time spent processing each kind of message received on the global events topic.

Audit logging has been improved.

The warning about unsaved changes being lost on an edited dashboard will now only show when actual changes have been made.

Fixed a bug in decryption code used when decrypting downloaded files from bucket storage when version-for-bucket-writes=3. The bug did not allow to decrypt files larger than 2GB.

Fixed an issue where LogScale could log secrets to the debug log when configured to use LDAP or when configured to use SSL for Kafka.

A new query function named createEvents() has been released. This function creates events from strings and is used for testing queries.

IP Location drilldowns now correctly use lat and lon field names instead of latitude and longitude .

URL paths with repository name and no trailing /search resolved to Not Found. The URL /repoName will now again show the search page for the repoName repository.

Bugs fixed for the collect() function, where:

Added a new configuration to control when digest coordination will permit a node that is not "in sync" and to set a time limit. See MAX_SECS_WAIT_FOR_SYNC_WHEN_CHANGING_DIGEST_LEADER for all the details.

Make adjustments to the HostsCleanerJob. It will now remove references to missing hosts in fewer writes, and will stop immediately if a host rejoins the cluster.

Fixed a bug seen in version 1.65 where groupBy() on multiple fields would sometimes produce multiple rows for the same combination of keys.

Fixed a race where unavailable segments, due to nodes going away, would not become available again after nodes returning.

Fixed an issue that could cause the error message Object is missing required member 'replicationFactor' when downgrading from current versions to older versions. The error message is only a nuisance, since the object failing deserialization isn't in use in released code yet.

Fixed an issue that could cause repeated unnecessary updates of currentHosts for some segments.

Fixed an issue where deleting a parser through an update or uninstall of a package could fail in an unexpected way if the parser was used by an ingest listener or an Fdr feed. Now, a proper error message will be shown.

The Repository icon has been changed to match the new look and feel.

A new UI for event forwarders located under Organisation Settings now allows you to configure your event forwarders. See Event Forwarders for details.

Added a query editor warning (yellow wavy lines) for joins in alerts.

Added a new dynamic configuration UndersizedMergingRetentionPercentage, with a default value of 20. This configuration value is used when selecting undersized segments to merge, this setting controls how wide a time span can be merged together.

The setting is interpreted as a percentage of the repository's retention by time setting. A reasonable range is 0 through to 90.

Increased the limits for bucket. The maximum number of series has been raised from 50 to 500 and the maximum number of output events has been raised from 10,000 to 100,000.

Avoid writing some messages to global if we can tell up-front that the message is unnecessary.

Reduce the scope of a precondition for a particular write to global. This should reduce unnecessary transaction rejections when such writes are bulked together.

Audit logging for s3 archiving which tracks when it is enabled, disabled, configured, and restarted.

Docker images have been upgraded to Java 17.0.5.

Fix some issues with the workings of the BUCKET_STORAGE_MULTIPLE_ENDPOINTS and S3_STORAGE_ENDPOINT_BASE configurations.

The intent of this configuration is to allow users to configure buckets in multiple bucket services, for instance to allow migrating from AWS bucket storage to a local S3 service. When true, each bucket in global can have a separate endpoint configuration, as defined in S3_STORAGE_ENDPOINT_BASE and similar configurations. This allows an existing cluster running against AWS S3 to begin uploading segments to an on-prem S3 by switching the endpoint base, while still keeping access to existing segments in AWS.

When false (default), the endpoint base configuration is applied to all existing buckets on boot. This is intended for cases where the base URL needs to be changed for all bucket, for instance due to the introduction of a proxy.

The issue was that we were not consistently looking up endpoint urls in global for the relevant bucket, but instead simply used whichever endpoint url happened to be defined in configuration at the time. This has been fixed.

Fixed a minor desynchronization issue related to idle datasources.

The SAML login to Humio using deeplinks now works correctly.

Fixed a bug where interaction context menus did not update the query editor in Safari.

When a host is removed from global, a job tries to clean up any references to it from other places in global, such as segments. Fixed a bug in this job that meant it didn't clean up references on segments that were tombstoned but not yet gone from global. This issue could block cleanup of those segments.

Added new dynamic configuration for MaxIngestRequestSize to allow limiting size of ingest requests after content-encoding has been applied. The default can be set using the new configuration variable MAX_INGEST_REQUEST_SIZE, or applied via the dynamic configuration.

It is now possible to specify a dashboard by name in the URL. It is also possible to have dashboard ID as a parameter in order to have permanent links.

Removed the restriction that case and match expressions cannot be used in subqueries.

The query function in() has been improved w.r.t. performance when searching in tag fields.

Improved memory allocation for the query function split().

The query function split() now allows splitting arrays that contain arrays. For example, the event a[0][0]=1, a[1][0]=2 can now be split using split(a) produces two events: _index=0, a[0]=1' and '_index=1, a[0]=2.

The query function join() now provides information to optimize the query.

In the internal request log, include decoded size of the request body after content-encoding has been applied in new field decodedContentLength. This allows inspecting compression ratio of incoming requests and range of values seen. Requests without compression have contentLength in this new field too.

Fixed a bug where a disabled item in the main menu could be clicked on and which would redirect to the homepage.

The Copy menu in the event Inspection Panel now copies text correctly again.

Fixed an issue where percentile() would crash on invalid input.

The query functions bucket(), holtwinters(), beta:repeating(), series(), session(), and window() wrongly accepted now as a duration. This error has been fixed.

Fixed an issue that could cause merged segments to appear to be missing after a restart, due to the datasource going idle.

Fixed an issue with API Explorer that could fail to load in some configurations when using cookie authentication.

Fixed an issue where the environment variable OIDC_USE_HTTP_PROXY was not respected. It means that LogScale will now call all OIDC endpoints directly without going through the HTTP Proxy, when OIDC_USE_HTTP_PROXY is set to false.

This fixes the known issue previously reported in Falcon LogScale 1.63.1 Stable (2022-11-14), Falcon LogScale 1.63.2 Stable (2022-11-30), Falcon LogScale 1.63.3 Stable (2022-12-21) and Falcon LogScale 1.70.0 Stable (2023-01-16).

Unlimited waits for nodes to get in sync has been fixed. This caused digest coordination to fail, to limit the time allowed for a node to get "in sync" on a partition before leadership was assigned to it, in cases where the previous digest leader shut down gracefully.

Add code to ensure all minisegments for the same target end up located on the same hosts. A change in 1.63 could create a situation where minisegments for the same merge target wound up on different nodes, which the query code currently assumes can't happen. This could cause Result is partial responses to user queries.

Fixed a bug where a link in the notification for a failed alert would link to a non-existing page.

Fixed a bug in decryption code used when decrypting downloaded files from bucket storage when version-for-bucket-writes=3. The bug did not allow to decrypt files larger than 2GB.

Fixed an issue where LogScale could log secrets to the debug log when configured to use LDAP or when configured to use SSL for Kafka.

OIDC known issue preventing login:

if you are running LogScale self-hosted, this version will not work in an environment where you are Authenticating with OpenID Connect and you use an HTTP Proxy, but you do not want to call OpenIDConnect through the HTTP Proxy.

Please do not upgrade to this version if this is the case — upgrade to Falcon LogScale 1.63.4 Stable (2023-02-01) and Falcon LogScale 1.70.1 Stable (2023-02-01) instead, where this issue has been fixed.

URL paths with repository name and no trailing /search resolved to Not Found. The URL /repoName will now again show the search page for the repoName repository.

OIDC known issue preventing login:

if you are running LogScale self-hosted, this version will not work in an environment where you are Authenticating with OpenID Connect and you use an HTTP Proxy, but you do not want to call OpenIDConnect through the HTTP Proxy.

Please do not upgrade to this version if this is the case — upgrade to Falcon LogScale 1.63.4 Stable (2023-02-01) and Falcon LogScale 1.70.1 Stable (2023-02-01) instead, where this issue has been fixed.

Added the new fileDownloadParallelism setting for FDR feeds to download files from the same SQS message in parallel. See Adjust Polling Nodes Per Feed for all the details.

Parsing JSON arrays in drill-down context menus no longer adds a trailing dot to the prefix field name.

Add Falcon LogScale announcement on login and signup pages.

Contextual drill-down menus for field interactions have been introduced, see Field Interactions. In particular:

Interactions on JSON data now enabled for JSON arrays in the Event List.

Humio is now a Falcon product. The Humio owl logo and icons are therefore replaced by beautiful falcons.

Following its name change, mentions of Humio have been changed to Falcon LogScale.

The Single Value widget has updated properties:

Change Humio logo to Falcon LogScale on login and signup pages.

Self-hosted only: the old implementation of how alert queries are run has been removed. As a consequence, the dynamic configuration UseLegacyAlertJob has also been removed.

Added two new message templates to actions, {query_start_s} and {query_end_s}. See Message Templates and Variables for details.

Added new createDashboardFromTemplateV2 mutation with input parameters aligned with the rest of the create from template mutations.

JSON in Log Line and JSON formats columns in Event List widgets now have fields underlined on hover and are clickable. This allows drill-downs and copying values easily.

QueryAPI — Added staticMetaData property to QueryJobStartedResult. At the moment it only contains the property executionMode, which can be used to communicate hints about the way the backend executes the query to the front-end.

QueryAPI — executionModeHint renamed to executionMode.

Introduced new valid array syntax in array:contains() and array:regex() functions:

Improved the format():

See the format() reference documentation page for all the above mentioned updates on the supported formatting syntax.

Added a new ingest endpoint for receiving metrics and traces via OpenTelemetry OTLP/http. See OpenTelemetry for all the details.

New background task that runs at startup. It verifies the checksums present in local segment files, traversing the most recently updated segment files on the local disk, using the timestamps they have when Humio status. If a file has invalid checksum it will be renamed to crc-error.X where X is the ID of the segment. An error will be logged as well.

Add an additional validation check when uploading files to S3-like bucket storage. Humio will now perform a HEAD request for the file's final location in the bucket to verify that the upload succeeded.

Created new test function for event forwarders, which takes as input an event forwarder configuration and tests whether it is possible to connect to the Kafka server. The current test function which takes an ID as input and tests an existing event forwarder by ID, is now marked as deprecated.

Added a new dynamic config UndersizedMergingRetentionPercentage, with a default value of 20. This configuration value is used when selecting undersized segments to merge, this setting controls how wide a time span can be merged together.

The setting is interpreted as a percentage of the repository's retention by time setting. A reasonable range is 0 through to 90.

Added use of the HTTP Proxy Client Configuration, if configured, in a lot of places.

When selecting a parser test case, the selected test case is highlighted in the UI, so you can see what is selected.

Empty datasource directories will be now removed from the local file system while starting the server.

Add a script in the tarball distribution's bin directory to check the execution environment, checking common permission issues and other requirements for an environment suitable for running Logscale.

It's now possible to expand multiple bell notifications.

Use latest version of Java 17 in Docker images.

Change missing @timestamp field to give a warning instead of an error in functions tail(), head(), bucket(), and timeChart().

Bug fixed in Scatter Chart widget tooltip, so that the description of the actual point only is shown in the tooltip when hovering the mouse over one point, instead of multiple points.

Fixed a bug where query result containing no valid results was handled incorrectly in visualisation.

Fixed an issue where NaN values could cause groupBy() queries to fail.

Fixed an issue where match() would sometimes give errors when ignoreCase=true and events contained latin1 encoded characters.

Fixed a bug where the selfJoin() function would not apply the postfilter parameter.

When selecting a parser test case, the selected test case is highlighted in the UI, so you can see what is selected.

It is now possible for a user to use the same personal invite token after the user has been transferred to another organization.

Fixed an issue where nothing was displayed on the average ingest chart in case only one datapoint is present.

When a host is removed from global, a job tries to clean up any references to it from other places in global, such as segments. Fixed a bug in this job that meant it didn't clean up references on segments that were tombstoned but not yet gone from global. This could block cleanup of those segments.

Fix an issue causing a content-length check for bucket uploads to fail when encryption was enabled. The content-length check is not normally enabled, so this should only affect clusters that have disabled ETag-based validation.

Fix an issue that could cause event redaction tasks to fail to complete, if a segment having events redacted was deleted due to retention.

Fixed a regression causing a reflective method lookup to fail when Humio is running on a Java prior to 13.

OIDC known issue preventing login:

if you are running LogScale self-hosted, this version will not work in an environment where you are Authenticating with OpenID Connect and you use an HTTP Proxy, but you do not want to call OpenIDConnect through the HTTP Proxy.

Please do not upgrade to this version if this is the case — upgrade to Falcon LogScale 1.63.4 Stable (2023-02-01) and Falcon LogScale 1.70.1 Stable (2023-02-01) instead, where this issue has been fixed.

Added enableEventForwarder and disableEventForwarder mutations to enable/disable event forwarders.

Humio Log Collector is now Falcon LogScale Collector.

New FleetOverview functionality for the LogScale Collector 1.2.0 is available.

LogScale Collector download page has moved into the new top level tab LogScale Collector Fleet Management (Cloud-only).

The base64Decode() query function now accepts non-canonical encodings.

The holtwinters() query function will be deprecated with the release of future version 1.68. From then, it cannot be expected to work in alerts, and it will be removed entirely with the release of version 1.72.

The parseCsv() function has improved its performance, in particular in terms of memory pressure.

Close all segments a node is working on when shutting down. This should help start later in Kafka after reboots.

Fixed a race condition where the segment top offset wasn't removed when a datasource went idle due to a race. This could result in event redaction not running for such segments.

Parsing JSON arrays in drill-down context menus no longer adds a trailing dot to the prefix field name.

Add Falcon LogScale announcement on login and signup pages.

Following its name change, mentions of Humio have been changed to Falcon LogScale.

Change Humio logo to Falcon LogScale on login and signup pages.

Introduced new valid array syntax in array:contains() and array:regex() functions:

Added a new ingest endpoint for receiving metrics and traces via OpenTelemetry OTLP/http. See OpenTelemetry for all the details.

Add a script in the tarball distribution's bin directory to check the execution environment, checking common permission issues and other requirements for an environment suitable for running Logscale.

Added the new fileDownloadParallelism setting for FDR feeds to download files from the same SQS message in parallel. See Adjust Polling Nodes Per Feed for all the details.

Interactions on JSON data now enabled for JSON arrays in the Event List.

QueryAPI — Added staticMetaData property to QueryJobStartedResult. At the moment it only contains the property executionMode, which can be used to communicate hints about the way the backend executes the query to the front-end.

QueryAPI — executionModeHint renamed to executionMode.

Fixed a bug where the selfJoin() function would not apply the postfilter parameter.

Fixed an issue where nothing was displayed on the average ingest chart in case only one datapoint is present.

Fix an issue that could cause event redaction tasks to fail to complete, if a segment having events redacted was deleted due to retention.

Fixed a regression causing a reflective method lookup to fail when Humio is running on a Java prior to 13.

Contextual drill-down menus for field interactions have been introduced, see Field Interactions. In particular:

Added new createDashboardFromTemplateV2 mutation with input parameters aligned with the rest of the create from template mutations.

JSON in Log Line and JSON formats columns in Event List widgets now have fields underlined on hover and are clickable. This allows drill-downs and copying values easily.

New background task that runs at startup. It verifies the checksums present in local segment files, traversing the most recently updated segment files on the local disk, using the timestamps they have when Humio status. If a file has invalid checksum it will be renamed to crc-error.X where X is the ID of the segment. An error will be logged as well.

It's now possible to expand multiple bell notifications.

Use latest version of Java 17 in Docker images.

Fixed a bug where query result containing no valid results was handled incorrectly in visualisation.

Fixed an issue where NaN values could cause groupBy() queries to fail.

The Single Value widget has updated properties:

Added two new message templates to actions, {query_start_s} and {query_end_s}. See Message Templates and Variables for details.

Humio is now a Falcon product. The Humio owl logo and icons are therefore replaced by beautiful falcons.

Improved the format():

See the format() reference documentation page for all the above mentioned updates on the supported formatting syntax.

Add an additional validation check when uploading files to S3-like bucket storage. Humio will now perform a HEAD request for the file's final location in the bucket to verify that the upload succeeded.

Empty datasource directories will be now removed from the local file system while starting the server.

Change missing @timestamp field to give a warning instead of an error in functions tail(), head(), bucket(), and timeChart().

Fix a regression introduced in 1.46.0 that can cause Humio to fail to properly replay data from Kafka when a node is restarted.

Fix an issue causing a content-length check for bucket uploads to fail when encryption was enabled. The content-length check is not normally enabled, so this should only affect clusters that have disabled ETag-based validation.

Humio is now a Falcon product. The Humio owl logo and icons are therefore replaced by beautiful falcons.

Created new test function for event forwarders, which takes as input an event forwarder configuration and tests whether it is possible to connect to the Kafka server. The current test function which takes an ID as input and tests an existing event forwarder by ID, is now marked as deprecated.

Added use of the HTTP Proxy Client Configuration, if configured, in a lot of places.

When selecting a parser test case, the selected test case is highlighted in the UI, so you can see what is selected.

Bug fixed in Scatter Chart widget tooltip, so that the description of the actual point only is shown in the tooltip when hovering the mouse over one point, instead of multiple points.

Fixed an issue where match() would sometimes give errors when ignoreCase=true and events contained latin1 encoded characters.

When selecting a parser test case, the selected test case is highlighted in the UI, so you can see what is selected.

It is now possible for a user to use the same personal invite token after the user has been transferred to another organization.

Fixed an issue where the HTTP threads (Akka pool) could get blocked while sending ingest requests to Kafka, which could result in Humio HTTP endpoints not responding.

Fixed a bug in decryption code used when decrypting downloaded files from bucket storage when version-for-bucket-writes=3. The bug did not allow to decrypt files larger than 2GB.

Fixed an issue where LogScale could log secrets to the debug log when configured to use LDAP or when configured to use SSL for Kafka.

The feature flag for FDR feeds has been removed. FDR feeds are now generally available.

The Log line format type in the Event List will now render fully expanded JSON when a JSON structure starts with a square bracket or curly bracket followed by a newline.

It is now possible to interact directly with the JSON properties and values in the EventList.

Humio is now a Falcon product. The Humio owl logo and icons are therefore replaced by beautiful falcons.

In the Event List you can assign data types to a column field. You can now make the setting the default for a fields and the setting is remembered when even the field is added to the Event List, e.g. from the fields panel on the Search page. The button for assigning default data type to a field can be found in the Data type dropdown menu in the column headers of the event list widget. See Field Data Types.

It is now possible to scroll to the selected event on the Search page.

The event lists column header menus have been redesigned to be simpler:

See Formatting Columns.

Fixed a broken link to the documentation for Message Templates and Variables when editing alerts and scheduled searches.

When you create or edit an action it will now show a warning dialog if you have unsaved changes.

A major change has been made to how alert queries are run in order to better reuse live queries when nodes are restarted in a Humio cluster. Find more details at Alerts.

When creating new Actions, the new name will now stay when you change the Action Type without getting cleared. This also works when you want to change the New Action name while creating a New Action.

Deprecates the defaultSharedTimeIsLive input field on the updateDashboard GraphQL mutation, in favor of updateFrequency.

New dynamic configuration MinimumHumioVersion, default value is 0.0.0, that allows setting a minimum Humio version that this cluster will accept starting on. This allows protecting against inadvertently later rolling back too far for some other feature to be turned on, that has an implied minimum version for support of that feature.

Added environment variable ENABLE_SANDBOXES to make it possible to enable and disable sandbox repositories.

On cloud: added a configuration on dynamic identity providers to configure if users are allowed to be lazily created.