Falcon LogScale Collector 1.2.0 GA (2022-10-27)

VersionTypeRelease DateConfig. Changes
1.2.0GA2022-10-27Yes
FileSHA256 Checksum
linux_amd64.deb34ebabca8336e6e84a97684dea8a6592eb893dc1db026096845fc1ff596996c3
linux_amd64.rpm2bcae521ba78bbdd54db0b8b77d536e80eca4a6bd1d3247e757e06ed424be93dd
linux_arm64.deb137f2d376a4d45045258ace6c8c7f9efb5bab808b67c195f98544862cbbf976a
linux_arm64.rpm93f3d1d37c86971ddf6e503e0832f361a98b34b93fecb3f92d696bb7d7355743
windows_amd64.msi9c47b0c008cd5ef83d5569132181d49c8ad929b59cf29e3a65a787bd88e9cce9

This version of the humio log collector offers the Fleet Overview functionality, which allows you to monitor the status of log collector instances and the following improvements:

  • Improved configuration file validation

  • Improved error logging

  • Reload configuration file feature

  • Using environment variables as the sink url

  • The file source now has more include and exclude patterns and uses less resources by waiting for changes to the file

  • the CMD source can now create single multiline events

  • the wineventlog can now filter events by provider and keep bookmarks of its progress

  • Performance improvements

  • improved batch handling

  • Enforces the use of HTTPS.

Improvements, new features and functionality

  • Configuration

    • Improved configuration file validation - The collector is now more thorough when validating its configuration file. An example of this is that unknown options in the configuration are invalid and will prevent running the program. Upon detection of an invalid configuration, the collecter will attempt to provide a descriptive error, some examples of this are: 

      error reading config file "my_config.yaml" sources:
 name must consist of only alphanumeric characters or '.', '_'
 and '-'

 error reading config file "my_config.yaml": 
 sources.cmd_uname_scheduled.interval: invalid type string,
 wanted int` 

 error reading config file "my_config.yaml"
 : sources.dummy_logs.sink: missing value for required field`

    • The collector now reloads the configuration file when it receives a SIGHUP. This does not apply to the logLevel and dataDirectory options. If the new configuration is invalid, the program will stop.

    • The collector now enforces using https:// for URLs, this can be overridden by adding the -allow-insecure-http command line flag.

  • Collecting Data

    • The url option in the sinks part of the configuration can now refer to an environment variable by using the ${ENVVAR} syntax.

    • The wineventlog source can now filter events based on the provider name. Set the option providers to an array of provider names that should be included to enable this feature. This source also keeps a bookmark of its progress in the Windows event log, and resumes from there when the collector is restarted.

    • Improved batch handling

      • The sinks now have additional configuration options to change the maximum event size maxEventSize (default 1MB) and the maximum batch size maxBatchSize (default: 16 MB). The limits are propagated to the queue, where it replaces the previous maxEventsPerRequest option. The limits are also propagated to all the sources that reference the sink.

      • The memory queue no longer supports configuration of maxEventsPerRequest, it inherits the maximum bytes per request from the sink maxBatchSize.

      • The memory queue no longer waits before flushing a batch that is larger than the maximum batch size.

      • The collector now warns you when a memory queue reaches 50% and 80% of capacity.

      • The collector now sends a warning after 2 retry attempts when sending events to a http sink.

    • The file source can now have additional include and exclude patterns in the same configuration. Specifically, the options exclude and include can be either a string or an array of strings.

    • The cmd source can now create a single multiline event when running in the schedule mode. Set the option consolidateOutput to true to enable this feature.

  • Fleet Overview

    • The collector now supports reporting to the fleet overview of LogScale. Configure the fleetManagement part of the configuration to enable this feature, see Fleet Management (fleet management) for more information.

      When the feature is enabled, the collector will periodically send metrics to LogScale, including the OS version, the collector version, how much data is ingested, and a description of the configured log sources.

  • Debugging

    • The default log level is now set to warnings, previously only errors were logged by default.

  • Managing Data

    • Improved serialization performance in the humiosink leading to lower memory usage and faster serialization of events.

    • Improved memory usage of the memory queue component by removing an upfront buffer that caused it to store more events than specified by the maxLimitInMB option.

    • If a file monitored by the file source is inactive (not written for a configurable period default: 60 seconds, the file descriptor is closed to release system resources, and watched for changes instead. Whenever the file changes, it is re-opened. This is configurable by the inactivityTimeout option in the file source.

Bug Fixes

  • Collecting Data

    • Events from the wineventlog source which contain fields of the type hexadecimal integer were presented as a base 10 number, they are now presented as a base 16 number.