Falcon LogScale 1.135.0 Preview (2024-04-23)

Version?Type?Release Date?Availability?End of Support

Security

Updates

Upgrades

From?

JDK

Compatibility?

Req. Data

Migration

Config.

Changes?
1.135.0Preview2024-04-23

Cloud

Next StableYes1.11217-21NoNo

Bug fixes and updates.

Advanced Warning

The following items are due to change in a future release.

Deprecation

Items that have been deprecated and may be removed in a future release.

  • The assetType GraphQL field on Alert, Dashboard, Parser, SavedQuery and ViewInteraction datatypes has been deprecated and will be removed in version 1.136 of LogScale.

  • The any argument to the type parameter of sort() and table() has been deprecated and will be removed in version 1.142.

    Warnings prompts will be shown in queries that fall into either of these two cases:

    • If you are explicitly supplying an any argument, please either simply remove both the parameter and the argument, for example change sort(..., type=any) to sort(...) or supply the argument for type that corresponds to your data.

    • If you are sorting hexadecimal values by their equivalent numerical values, please change the argument of type parameter to hex e.g. sort(..., type=hex).

    In all other cases, no action is needed.

    The new default value for sort() and table() will be number. Both functions will fall back to lexicographical ordering for values that cannot be understood as the provided argument for type.

  • The following API endpoints are deprecated and marked for removal in 1.148.0:

    • POST /api/v1/clusterconfig/kafka-queues/partition-assignment

    • GET /api/v1/clusterconfig/kafka-queues/partition-assignment

    • POST /api/v1/clusterconfig/kafka-queues/partition-assignment/set-replication-defaults

    The deprecated methods are used for viewing and changing the partition assignment in Kafka for the ingest queue. Administrators should use Kafka's own tools for editing partition assignments instead, such as the bin/kafka-reassign-partitions.sh and bin/kafka-topics.sh scripts that ship with the Kafka install.

  • The following GraphQL queries and mutations for interacting with parsers are deprecated and scheduled for removal in version 1.142.

    • Deprecated createParser mutation is replaced by createParserV2(). The differences between the old and new mutation are:

      • testData input field is replaced by testCases, which can contain more data than the old tests could. This includes adding assertions to the output of a test. These assertions are not displayed in the UI yet. To emulate the old API, you can take the old test string and put it in the ParserTestEventInput inside the ParserTestCaseInput, and they will behave the same as before.

      • fieldsToBeRemovedBeforeParsing can now be specified as part of the parser creation.

      • force field is renamed to allowOverwritingExistingParser.

      • sourceCode field is renamed to script.

      • tagFields field is renamed to fieldsToTag.

      • languageVersion is no longer an enum, but a LanguageVersionInputType instead.

      • The mutation returns a Parser, instead of a Parser wrapped in an object.

      • The mutation fails when a parser has more than 2,000 test cases, or the test input in a single test case exceeds 40,000 characters.

    • Deprecated removeParser mutation is replaced by deleteParser. The difference between the old and new mutation is:

      • The mutation returns boolean to represent success or failure, instead of a Parser wrapped in an object.

    • Deprecated testParser mutation is replaced by testParserV2(). The differences between the old and new mutation are:

      • The test cases are now structured types, instead of just being strings. To emulate the old API, take the test string and put it in the ParserTestEventInput inside the ParserTestCaseInput, and they will behave the same as before.

      • The new test cases can contain assertions about the contents of the output.

      • The mutation output is significantly different from before, as it provides more detailed information on how a test case has failed.

      • The mutation now accepts both a language version and list of fields to be removed before parsing.

      • The parserScript field is renamed to script.

      • The tagFields field is renamed to fieldsToTag.

    • Deprecated updateParser mutation is replaced by updateParserV2(). The differences between the old and new mutation are:

      • testData input field is replaced by testCases, which can contain more data than the old tests could. This includes adding assertions to the output of a test. These assertions are not displayed in the UI yet. To emulate the old API, you can take the old test string and put it in the `ParserTestEventInput` inside the `ParserTestCaseInput`, and they will behave the same as before.

      • sourceCode field, used to updating the parser script, is changed to the script field, which takes a UpdateParserScriptInput object. This updates the parser script and the language version together.

      • tagFields field is renamed to fieldsToTag.

      • The languageVersion is located inside the UpdateParserScriptInput object, and is no longer an enum, but a LanguageVersionInputType instead.

      • The repositoryName and id fields are now correctly marked as mandatory in the schema. Previously this wasn't the case, even though the mutation would fail without them.

      • The mutation returns a Parser, instead of a Parser wrapped in an object.

      • The old mutation had a bug where it would overwrite the languageVersion with a default value in some cases, which is fixed in the new one.

      • The mutation fails when a parser has more than 2,000 test cases, or the test input in a single test case exceeds 40,000 characters.

    On the Parser type:

    • testData field is deprecated and replaced by testCases.

    • sourceCode field is deprecated and replaced by script.

    • tagFields field is deprecated and replaced by fieldsToTag.

    For more information, see Parser, DeleteParserInput, LanguageVersionInputType, createParserV2(), testParserV2(), updateParserV2().

  • In the GraphQL API, the ChangeTriggersAndAction enum value for both the Permission and ViewAction enum is now deprecated and will be removed in version 1.136 of LogScale.

  • The HUMIO_JVM_ARGS environment variable in the Launcher Script script will be removed in 1.154.0.

    The variable existed for migration from older deployments where the launcher script was not available. The launcher script replaces the need for manually setting parameters in this variable, so the use of this variable is no longer required. Using the launcher script is now the recommended method of launching LogScale. For more details on the launcher script, see Launcher Script. Clusters that still set this configuration should migrate to the other variables described at Override garbage collection configuration within the launcher script.

  • We are deprecating the humio/kafka and humio/zookeeper Docker images due to low use. The planned final release for these images will be with LogScale 1.148.0.

    Better alternatives are available going forward. We recommend the following:

    If you still require humio/kafka or humio/zookeeper for needs that cannot be covered by these alternatives, please contact Support and share your concerns.

  • In the GraphQL API, the name argument to the parser field on the Repository datatype has been deprecated and will be removed in version 1.136 of LogScale.

Upgrades

Changes that may occur or be required during an upgrade.

  • Storage

    • Docker images have been upgraded to Java 22.

    • Added new deployment artifacts. The published tarballs (e.g. server.tar.gz) are now available with a bundled JDK. The platforms currently supported are linux_x64 for 64-bit Linux, and alpine_x64 for 64-bit Alpine Linux and other musl-based Linuxes. The Docker images have been updated to use this bundled JDK internally. We encourage users to migrate to using the tarballs with bundled JDKs.

Improvements, new features and functionality

  • UI Changes

    • The Query Editor now shows completions for known field values that have previously been observed in results. For instance, #repo = m may show completions for repositories starting with m seen in previous results.

  • Automation and Alerts

  • Storage

    • We reverted a change introduced in 1.131.0 intended to cause fewer minisegments to move in the cluster when digest reassignment occurs. The change could cause minisegments to not be balanced across cluster nodes in the expected way.

  • Configuration

    • The default values for the following configuration parameters have changed:

      • FLUSH_BLOCK_SECONDS = 900 (was 1,800)

      • MAX_HOURS_SEGMENT_OPEN = 720 (was 24, maximum is now 24,000)

    • The amount of global meta data required for retention spans of over 30 days has been reduced. The amount of global meta data required in clusters with high number of active datasources has also been reduced, as well as the global size of mini segments, by combining them into larger mini segments.

      Pre-merging mini segments now reduces the number of segment files on disk (and in bucket) and reduces the amount of meta data for segment targets in progress. This allows getting larger target segment files and reduces the amount of "undersized" merging of "completed" segments. It also allows a smaller flush interval for mini segments without incurring in a larger number of mini segments.

      This feature is only supported from v1.112.0. To safely enable it by default, we are now raising to v1.112.0 the minimum version to upgrade from, to disallow rollback to versions older than this version.

      The feature is on by default. It can be disabled using the feature flag PreMergeMiniSegments. Disabling the feature stops future merges of mini segments into larger mini segment files, but does not alter the defaults below, nor modify how already merged minisegments behave.

      For more information, see Global Database, Ingestion: Digest Phase.

    • The following configuration parameters have been introduced:

  • Dashboards and Widgets

    • Dashboard parameters have gotten the following updates:

      • The name of the parameter is on top of the input field, so more space is available for both parts.

      • A Clear all button has been added to multi-value parameters so that all values can be removed in one click.

      • The parameter's configuration form has been moved to the side panel.

      • Multiple values can be added at once to a multi-value parameter by inputting a comma separated list of values, which can be used as individual values.

      For more information, see Multi-value Parameters.

    • The automatic rendering of URLs as links has been disabled for the Table widget. Only URLs appearing in queries with the markdown style e.g. [CrowdStrike](https://crowdstrike.com) will be automatically rendered as links in the Table widget columns. Content, including plain URLs e.g. https://crowdstrike.com, can still be rendered as links, but this should now be explicitly configured using the Show asLink widget property.

      For more information, see Table Widget Properties.

  • Functions

    • onDuplicate parameter has been added to kvParse() to specify how to handle duplicate fields.

Bug Fixes

  • Dashboards and Widgets

    • Other options than exporting to CSV file were not possible on the Dashboard page for a widget and on the Search page for a query result. This issue is now fixed.

  • Functions

    • The error message when providing a non-existing query function in an anonymous query e.g. bucket(function=[{_noFunction()}]) has been fixed.