Falcon LogScale Collector 1.2.0 GA (2022-10-27)
|Version||Type||Release Date||Config. Changes|
This version of the humio log collector offers the Fleet Overview functionality, which allows you to monitor the status of log collector instances and the following improvements:
Improved configuration file validation
Improved error logging
Reload configuration file feature
Using environment variables as the sink url
The file source now has more include and exclude patterns and uses less resources by waiting for changes to the file
the CMD source can now create single multiline events
the wineventlog can now filter events by provider and keep bookmarks of its progress
improved batch handling
Enforces the use of HTTPS.
Improvements, new features and functionality
Improved configuration file validation - The collector is now more thorough when validating its configuration file. An example of this is that unknown options in the configuration are invalid and will prevent running the program. Upon detection of an invalid configuration, the collecter will attempt to provide a descriptive error, some examples of this are:
error reading config file "my_config.yaml" sources: name must consist of only alphanumeric characters or '.', '_' and '-' error reading config file "my_config.yaml": sources.cmd_uname_scheduled.interval: invalid type string, wanted int` error reading config file "my_config.yaml" : sources.dummy_logs.sink: missing value for required field`
The collector now reloads the configuration file when it receives a
SIGHUP. This does not apply to the
dataDirectoryoptions. If the new configuration is invalid, the program will stop.
The collector now enforces using
https://for URLs, this can be overridden by adding the -allow-insecure-http command line flag.
urloption in the sinks part of the configuration can now refer to an environment variable by using the
wineventlogsource can now filter events based on the provider name. Set the option
providersto an array of provider names that should be included to enable this feature. This source also keeps a bookmark of its progress in the
Windows event log, and resumes from there when the collector is restarted.
Improved batch handling
The sinks now have additional configuration options to change the maximum event size
maxEventSize(default 1MB) and the maximum batch size
maxBatchSize(default: 16 MB). The limits are propagated to the queue, where it replaces the previous
maxEventsPerRequestoption. The limits are also propagated to all the sources that reference the sink.
The memory queue no longer supports configuration of
maxEventsPerRequest, it inherits the maximum bytes per request from the sink
The memory queue no longer waits before flushing a batch that is larger than the maximum batch size.
The collector now warns you when a memory queue reaches 50% and 80% of capacity.
The collector now sends a warning after 2 retry attempts when sending events to a http sink.
filesource can now have additional
excludepatterns in the same configuration. Specifically, the options
includecan be either a string or an array of strings.
cmdsource can now create a single multiline event when running in the schedule mode. Set the option
consolidateOutputto true to enable this feature.
The collector now supports reporting to the fleet overview of LogScale. Configure the
fleetManagementpart of the configuration to enable this feature, see Fleet Management (
fleet management) for more information.
When the feature is enabled, the collector will periodically send metrics to LogScale, including the OS version, the collector version, how much data is ingested, and a description of the configured log sources.
The default log level is now set to warnings, previously only errors were logged by default.
Improved serialization performance in the
humiosink leading to lower memory usage and faster serialization of events.
Improved memory usage of the memory queue component by removing an upfront buffer that caused it to store more events than specified by the
If a file monitored by the
filesource is inactive (not written for a configurable period default: 60 seconds, the file descriptor is closed to release system resources, and watched for changes instead. Whenever the file changes, it is re-opened. This is configurable by the
inactivityTimeoutoption in the file source.
Events from the
wineventlogsource which contain fields of the type hexadecimal integer were presented as a base 10 number, they are now presented as a base 16 number.