Full Falcon Log Collector Release Notes Index

This section contains a single page with all release notes on the same page.

Falcon Log Collector 1.8.1 GA (2024-11-20)
Version?Type?Release Date?

Config.

Changes?
1.8.1GA2024-11-20no

Hide file hashes

Show file hashes

Highlights

Support for including or excluding events based on regular expressions has been introduced.

Behavior Changes

Scripts or environment which make use of these tools should be checked and update for the new configuration:

  • This release requires at least Windows 10 or Windows Server 2016.

  • Users running Falcon Log Collector version 1.3.4 or earlier must first update to version 1.6.6 before updating to version 1.8.1.

Improvements, new features and functionality

  • Collecting Data

    • Falcon Log Collector now supports filtering events based upon a regular expression. Filtering is available per source and is configured using a transform with type regex_filter. Two modes are available. Include, which will include only events matching a regex, and exclude, which will exclude all events matching a regex. For more information, see Configuration Elements.

    • Improved the syslog source for TCP/TLS by removing the 1024 connection limit, further enhancing the memory optimization introduced in 1.7.4.

    • The checkpointer has been optimized for improved performance and scalability with multiple file sources. This change includes a revised internal database structure, which affects how file identities are stored. An automatic migration is performed during upgrade, preserving existing file identities.

      Warning

      Downgrading from 1.8.0 to an older version, the collector will not be able to recognize the checkpoints for files identified in the new format, potentially causing re-ingestion. For more information, see How-To: Downgrading LogScale Collector from Version 1.8.1 to 1.7.x.

  • Debugging

    • Internal log messages in the Falcon Log Collector have been improved. Some trace level messages regarding memory queue handling have been removed and some debug level messages regarding collector metrics have been added.

  • Other

    • To take advantage of the latest optimizations and security updates the Go version has been updated. With the latest update this requires at least Windows 10 or Windows Server 2016; support for previous versions has been discontinued.

    • The UserAgent string used in HTTP requests has been updated and aligned across platforms, it now includes OS and architecture.

  • Installation and Deployment

    • The install scripts have been updated to replace legacy launchctl subcommands load and unload with newer commands in macOS and to install the Falcon Log Collector as a service in macOS,

Bug Fixes

  • Other

    • Windows only: Fixed a bug where the collector would fail to start after increasing the maxLimitInMB parameter for an already configured disk queue, due to a file rename error when re-allocating the disk memory storage file.

  • Fleet Management

    • We have identified some edge cases in which a reconfiguration of a sink could cause the error: "Could not send data to sink. Sending will be retried, context canceled" to be reported even though subsequent transmission has succeeded.

    • If a Falcon Log Collector instance encounters an error while sending data to a sink this error will be reported in Fleet Management and the status column in the Fleet Overview page will display error. If a subsequent transmission succeeds, status will return to ok.

    • A previously unhandled scenario where the Falcon Log Collector attempts to enroll into Fleet management would block indefinitely due to a missing HTTP response has been addressed. The Collector now times out after 60 seconds and logs a warning: "timeout awaiting response headers".

  • Installation and Deployment

    • The uninstall script for linux distributions would fail to remove the service user on RedHat distributions.

Falcon Log Collector 1.7.4 GA (2024-10-03)
Version?Type?Release Date?

Config.

Changes?
1.7.4GA2024-10-03no

Hide file hashes

Show file hashes

Performance improvements for file sources and syslog sources.

Improvements, new features and functionality

  • Collecting Data

    • File source optimization for Windows and macOS.

      The filesource keeps monitoring files after all data has been read and ingested to be able to continue shipping when/if new data is added. In scenarios with a high number of files, this can be rather CPU consuming on Windows and macOS. In order to reduce the CPU usage a dynamic file scanner, which balances the CPU usage of the file scanning part has been introduced.

  • Debugging

    • Internal log messages in the LogScale Collector have been improved. An internal buffer has been increased to avoid missing internal logs and log messages for the syslog source have been augmented with more detail.

  • Other

    • Symbol names and debug information are now stripped from binaries, this results in smaller distributables.

  • Fleet Management

    • The LogScale Collector now supports sending custom labels to Fleet Management.

      This is in preparation for an upcoming Fleet Management feature which allows using labels for defining collector groups. When creating a group - labels can be used in the filter query. E.g. labels.myLabel=foo. Labels must be added to the local fleet management config file of the LogScale Collector. Label values can be expanded from environment variables as well.

  • Installation and Deployment

    • The syslog source has been optimized to use less memory in setups with high amounts of short lived TCP connections. The new approach utilizes a memory pool instead of allocating a new memory for each connection.

      The number of concurrent TCP connections is limited to 1024. The default MaxEventSize for syslog over TCP is changed to 1 MB to match the same setting when using syslog over TLS.

      The previous default setting was 2048 B (which is mentioned in the RFC), however as some users have experienced truncated events, the setting has been changed.

Bug Fixes

  • Debugging

    • A race condition related to file rotation using compression could cause the checkpointer to get in a state where it would repeatedly log the following error messages, "File failed, waiting 1min. error: EOF" and "pipeline failed, error: EOF".

      The LogScale Collector now marks the checkpoint for the file as done and the warning message "Handling unexpected EOF in compressed file" will be logged once.

  • Other

    • We have identified a previously unhandled scenario in which the LogScale Collector attempts to send data to a Data Connector and either a HTTP intermediary, such as a proxy, or the Data Connector accepts the HTTP connection, however never returns a HTTP response and in the same time keeps the connection alive.

      Previously this would cause the LogScale Collector to wait for the response, thus blocking further data on that sink. To address this scenario, the LogScale Collector will now timeout, and attempt re-transmission, if it does not receive a response within 60 s. In this case a warning: "timeout awaiting response headers" will be logged.

Falcon Log Collector 1.7.3 GA (2024-08-13)
Version?Type?Release Date?

Config.

Changes?
1.7.3GA2024-08-13no

Hide file hashes

Show file hashes

Bugfix and improved support of uninstall.

Improvements, new features and functionality

  • Installation and Deployment

    • Uninstall-scripts are now distributed as part of the "Full Install" via bash / powershell.

      Note that uninstalling a custom installation via system packages (i.e. rpm, deb, msi) is still handled by the system's package manager.

      See Uninstall Falcon Log Collector Installations for more information on how to uninstall LogScale Collector.

Bug Fixes

  • Other

    • A vulnerability in a library generating UUIDs is addressed. The UUIDs were exclusively used for non-cryptographic purposes.

Falcon Log Collector 1.7.2 GA (2024-07-09)
Version?Type?Release Date?

Config.

Changes?
1.7.2GA2024-07-09no

Hide file hashes

Show file hashes

Bugfix for the file source (Linux only).

Improvements, new features and functionality

  • Collecting Data

    • The LogScale Collector (Linux only) was leaking file handles when monitoring symlinked files, causing the file scanning to be reduced to 10 second intervals. This update ensures that the file handles are closed when the symlink or the files are moved or deleted.

Falcon Log Collector 1.7.1 GA (2024-06-27)
Version?Type?Release Date?

Config.

Changes?
1.7.1GA2024-06-27no

Hide file hashes

Show file hashes

Bug fixes to 1.7.0 regarding the file source and Fleet Management config changes.

Improvements, new features and functionality

  • Collecting Data

    • For Kubernetes deployments, the kubernetes transform (used by the Helm chart to collect Pod logs) now also collects metadata for init containers.

Bug Fixes

  • Collecting Data

    • Linux only: Fixed a bug in the inotify event handler which would cause excessive error messages in the internal logs.

    • Fixed a bug in the file source which potentially caused the collector to stop reading files, related to a race between the file inactivity timeout and a file modify event.

  • Fleet Management

    • Fixed an issue where the LogScale Collector could stop ingesting after publishing config changes from Fleet Management.

Falcon Log Collector 1.7.0 GA (2024-06-03)
Version?Type?Release Date?

Config.

Changes?
1.7.0GA2024-06-03no

Hide file hashes

Show file hashes

Download

  • Docker EU-1 registry.crowdstrike.com/log-collector/eu-1/release/logscale-collector:1.7.0

  • Docker US-1 registry.crowdstrike.com/log-collector/us-1/release/logscale-collector:1.7.0

  • Docker US-2 registry.crowdstrike.com/log-collector/us-2/release/logscale-collector:1.7.0

  • Support for ephemeral hosts

  • Performance improvements to the file source on linux, the windows Event source and general memory handling

Improvements, new features and functionality

  • Collecting Data

    • The LogScale Collector has been optimised for a more deterministic memory footprint. Memory is now reserved in the queue before reading from each source. This will reduce memory usage in e.g. backfill scenarios with a high number of individual files.

    • Linux only: To reduce the CPU and file I/O usage, the file source now utilises inotify for monitoring file changes.

    • The Winevent log source now supports severity filters and custom XPath and XML queries. The severity filter can be used to only include events with specific levels, e.g.adding the key levels: [0,1,2,3] to the channel specification will only include events with levels above 4 (information). The queries can be used to build more specific filters.

  • Debugging

    • Cleaned up the internal logs messages in the LogScale Collector. Some lowered in severity and some removed

    • The internal logging component handles more events per second to eliminate the "Dropped debug log.." message.

  • Other

    • The backward compatibility checkpoint.json is from this release obsolete. If migrating from a version before 1.4.0 to 1.7.0 and above, you need to install and run 1.6.5, in order to preserve the checkpoints.

  • Fleet Management

    • Added support for ephemeral mode by specifying an ephemeral timeout at enrollment. If a collector is offline for the specified duration, it will be unenrolled and disappear from the fleet overview.

    • Added auto enrollment functionality that automatically enrolls the LogScale Collector if it does not have a working access token. Refer to Fleet Modes on how to use this feature.

Falcon Log Collector 1.6.6 GA (2024-06-13)
Version?Type?Release Date?

Config.

Changes?
1.6.6GA2024-06-13no

Hide file hashes

Show file hashes

Bug fix related to the handling of the file source.

Bug Fixes

  • Collecting Data

    • Fixed a memory leak in the file source. In some scenarios the process would not release memory after a file was closed.

  • Debugging

    • Added User-Agent header to debug logging component.

Falcon Log Collector 1.6.5 GA (2024-04-29)
Version?Type?Release Date?

Config.

Changes?
1.6.5GA2024-04-29no

Hide file hashes

Show file hashes

Download

  • Docker EU-1 registry.crowdstrike.com/log-collector/eu-1/release/logscale-collector:1.6.5

  • Docker US-1 registry.crowdstrike.com/log-collector/us-1/release/logscale-collector:1.6.5

  • Docker US-2 registry.crowdstrike.com/log-collector/us-2/release/logscale-collector:1.6.5

Improvements and bug fixes regarding handling of the file source.

Improvements, new features and functionality

  • Debugging

    • On Windows, when file open fails, the error message will now include the path to the file, matching the behaviour on Linux and macOS.

    • The error message "Could not identify file." will only be logged once per file that could not be identified. Previously this message would be logged at each failed attempt to identify the file.

    • The severity of a number of internal LogScale Collector log messages has been reduced from error to warning, in cases where the error is already handled.

    • The error message "Failed sink receive." was previously erroneously logged at LogScale Collector shutdown, in scenarios where the shutdown was intentional and graceful. This has been corrected.

Bug Fixes

  • Collecting Data

    • A race condition during file rotation potentially causing the LogScale Collector to crash has been identified and addressed.

    • In previous versions the file source would hold on to a file handle until the read content was successfully received by LogScale. This could cause the reported disk usage to be higher than expected in scenarios with frequent file-rotation and low bandwidth/loss of connection to LogScale. The LogScale Collector now releases file handles immediately after reading EOF or detecting files being removed.

    • A race condition which could cause the LogScale Collector to crash, when using the static_fields transform, has been identified and addressed.

  • Debugging

    • An issue causing the internal debug logging module to log a warning "Dropped debug log message before they enter the ring buffer" has been fixed.

  • Fleet Management

    • Fixed a bug where LogScale Collector metrics would fail to run, when any of the metrics could not be collected.

Falcon Log Collector 1.6.2 GA (2024-02-26)
Version?Type?Release Date?

Config.

Changes?
1.6.2GA2024-02-26yes

Hide file hashes

Show file hashes

Download

  • Docker EU-1 registry.crowdstrike.com/log-collector/eu-1/release/logscale-collector:1.6.2

  • Docker US-1 registry.crowdstrike.com/log-collector/us-1/release/logscale-collector:1.6.2

  • Docker US-2 registry.crowdstrike.com/log-collector/us-2/release/logscale-collector:1.6.2

Improvements and bugfixes regarding handling of the file source and installation on Windows.

Improvements, new features and functionality

  • Collecting Data

    • The file source now supports environment variable expansions in the include/exclude field in the config.

Bug Fixes

  • Collecting Data

    • An requirement has been removed where in previous versions the Windows Event Log source required the existence of at least one channel.

    • An issue has been resolved when the checkpointing component used by the file source. If a file was rotated/moved while the content was read but not yet acknowledged through the network, the unacknowledged content could be re-transmitted, resulting in duplication of ingest to LogScale.

  • Other

    • Fixed a bug where the --allow-insecure-http flag would not take effect for communication with Fleet Management. This caused the collector to enforce using https:// for Fleet Management communication even if the enroll command was executed with the --allow-insecure-http flag.

  • Installation and Deployment

    • Installing the LogScale Collector on Windows using the Windows installer MSI will install it as a service and set the service start setting to Manual. After enrolling the Collector into Fleet Management using the enroll command, the service start setting will be set to to Automatic. and the service will be started.

      In previous versions of the installer a new install/upgrade using the installer MSI would set the service start setting to Manual. (overriding any current setting), as of 1.6.2 the installer service start setting will be preserved unless you do a fresh install.

      When performing a new enroll command, You will set the start to Automatic., therefore if you want to set to a custom setting, e.g. Automatic (Delayed start). this will need to be done after the enroll command is performed.

      Note

      downgrading to version 1.6.1 or earlier will still revert to the initial setting (Manual.).

Falcon Log Collector 1.6.1 GA (2023-12-12)
Version?Type?Release Date?

Config.

Changes?
1.6.1GA2023-12-12yes

Hide file hashes

Show file hashes

Download

  • Docker EU-1 registry.crowdstrike.com/log-collector/eu-1/release/logscale-collector:1.6.1

  • Docker US-1 registry.crowdstrike.com/log-collector/us-1/release/logscale-collector:1.6.1

  • Docker US-2 registry.crowdstrike.com/log-collector/us-2/release/logscale-collector:1.6.1

Improvements and bugfixes to the handling of the file source and debug logging.

Improvements, new features and functionality

  • Collecting Data

    • The file source been improved in scenarios with log file rotation. In previous versions a race could occur between file discovery and file read, during file rotation. This could potentially cause data to be missed (not ingested).

    • Performance of Syslog UDP source has been improved when running on a Linux system.

      The source now uses multiple workers to receive data from the network. By default it will spawn workers corresponding to the number of CPU cores in the system. The number of workers can be controlled by specifying the 'workers' parameter under the source configuration. Specifying 0 workers, or omitting the parameter will use the detected number of cores. See Sources & Examples.

    • We now offer a docker image which can be deployed in a Kubernetes to forward log messages from the applications deployed in the cluster. See te documentation for more details Collect Kubernetes Pod Logs.

  • Debugging

    • Debug logging now uses the system proxy by default, previously proxy was not supported.

    • The debug logging configuration through environment variables now supports additional configuration options which are documented in Debug Log

    • Debug logging no longer accepts http:// addresses by default. If such a scenario is required then HUMIO_DEBUG_LOG_ALLOW_HTTP must be set to true.

    • The debug logging mechanism has been improved. If an error occurs when sending the debug logs to LogScale, 3 attempts will be made in total before the debug logs are dropped.

Bug Fixes

  • Collecting Data

    • Configuring an incorrect exclude path for the file source could cause the collector to crash, this is corrected.

Falcon Log Collector 1.5.3 GA (2023-10-16)
Version?Type?Release Date?

Config.

Changes?
1.5.3GA2023-10-16no

Hide file hashes

Show file hashes

Improvements and bugfixes regarding handling of the file source.

Improvements, new features and functionality

  • Collecting Data

    • Internal handling of the file source has been optimized to reduced the number of syscalls used to monitor files and directories for changes.

Bug Fixes

  • Collecting Data

    • An issue has been identified with the checkpointing component, causing unnecessary disk write operations when used with a file source. This scales linearly with the number of file sources configured, leading to a high disk utilization when defining multiple file sources.

      Note: Multiple `include` patterns per file source do not cause this issue, only separate sources. This behaviour started in 1.5.0, and is fixed with this release.

Falcon Log Collector 1.5.2 GA (2023-10-03)
Version?Type?Release Date?

Config.

Changes?
1.5.2GA2023-10-03no

Hide file hashes

Show file hashes

Bug fixes and improvements.

Improvements, new features and functionality

  • Collecting Data

    • When LogScale rejects an ingest API request due to a request timeout or the request being too large, the LogScale Collector now divides the ingest request in to multiple parts and attempts to send the split files. If after dividing the ingest request, if a single event still triggers this limit, it will be discarded.

      The default LogScale request size limit is 32 MB, while the LogScale Collector targets maximum of 16 MB of input per request. Due to encoding, particularly control characters or invalid UTF-8 sequences could cause an up to 6x blow up of the request size.

    • Added a logscale alias for the humio sink. It is now possible to write type: logscale instead of type: humio in the sinks section.

  • Debugging

    • The log level for the log message "File is a duplicate of another file." has been changed from warning to info.

Bug Fixes

  • Collecting Data

    • Fixed a bug where an invalid include/exclude pattern in the config of a file source could cause the LogScale Collector to crash.

    • Fixed a bug where a duplicate of a file could trigger length updates in the open file source.

      If a duplicate file is an included file that has the same fingerprint as another included file. The lexicographically lesser path is considered the active file.

    • Fixed a bug when inadvertently reading a binary file could induce a 400 Bad Request from LogScale, which discards data in the LogScale Collector.

      The issue occurs when a binary file contains a UTF-8 sequence of EF BF BF that decodes to U+FFFF. The U+FFFF code point gets interpreted as end-of-input in the applicable LogScale ingest API.

    • The file source now completely ignores files that are of length zero bytes. This should fix an issue where the file source would inadvertently read a compressed file as plain text, if the file was opened when it was empty.

      This scenario is most likely to occur when a log file is rotated and compressed. Reading a compressed file as plain text could then induce the above binary file problem regarding U+FFFF.

Falcon Log Collector 1.5.1 GA (2023-8-28)
Version?Type?Release Date?

Config.

Changes?
1.5.1GA2023-8-28no

Hide file hashes

Show file hashes

Bug fixes.

Bug Fixes

  • Collecting Data

    • Fixed a bug which caused the Log Collector to crash when using multiple file source declarations.

Falcon Log Collector 1.5.0 GA (2023-8-23)
Version?Type?Release Date?

Config.

Changes?
1.5.0GA2023-8-23yes

Hide file hashes

Show file hashes

  • The LogScale Collector now supports macOS and is available as package installer (.pkg).

  • The LogScale Collector now reports metrics regarding e.g. CPU and memory usage to LogScale Fleet Management.

Improvements, new features and functionality

  • Collecting Data

    • The syslog source has been optimized with respect to UDP mode. According to internal performance measurements, the performance has been increased by a factor of 3-4.

    • The file source has been updated with improved file identity tracking. If multiple files are considered to be identical copies through fingerprinting, only a single copy will be opened.

    • The LogScale Collector now supports macOS and is available as package installer (.pkg), see Install Falcon Log Collector for information.

      The installer contains a universal binary which runs natively on both Apple silicon and Intel-based Mac computers. In addition to the source types supported on other platforms e.g. file source etc., a new source type unifiedlog has been added, see Sources & Examples. This source type supports shipping unified logs on macOS.

    • The disk queue has been reimplemented in order to increase performance and resilience.

      One consequence of this is that the entire storage space, determined by maxLimitInMB, is allocated when the queue is created. This ensures a deterministic size of the disk queue and prevents scenarios where the configured disk queue size is not available due to missing disk space.

      If the configured disk queue size is not available on the configured disk partition, an error will be issued. E.g. "Could not apply the config error="pipeline: logscale, details: no space left on device"".

    • If two instances of the LogScale Collector are attempting to use the same data directory, the error message has been improved.

      An example scenario is if the Collector is running as a service and a second instance is started manually from the command line. Previously the error message would be: timeout.

      Now the following error message will be issued: "Could not lock the checkpoint database. Maybe another process is using the same data directory? The data directory is set to: my_data_directory_location"

    • A new source type syslog_tls has been added. This source type supports receiving encrypted syslog traffic. See Sources & Examples for more information.

  • Fleet Management

    • The LogScale Collector will now send its CPU usage, memory usage and disk usage of the data directory partition to LogScale Fleet Management.

      These metrics will be available from within the Fleet Management|Fleet Overview pagein the LogScale user interface and can be used to provide a feedback loop when scaling instances and adjusting configuration settings. See Falcon Log Collector Metrics for more information.

Falcon Log Collector 1.4.1 GA (2023-6-13)
Version?Type?Release Date?

Config.

Changes?
1.4.1GA2023-6-13yes

Hide file hashes

Show file hashes

Improvements to the handling of the Windows event log source.

Improvements, new features and functionality

  • Collecting Data

    • The approach for handling Windows Event Logs has been revised, as the previous versions of the Collector could cause field names to be misaligned.

      The previous approach was solely based on using the Windows Event API for rendering the field names. This has shown to fail in cases where the event data has a parameter without a value.

      The new approach parses the XML and for events containing EventData, the field names and values are directly extracted from the XML. For events containing UserData, the XML may not be sufficient, thus the parsing falls back to the Windows Event API to render the field names.

      This has the following known impact on the collector data:

      • Corrects the misalignment of field names, found in earlier versions.

      • Events containing the Binary field, are now sent as their real names, e.g, windows.EventData.Binary, which previously were sent as windows.EventData[n].

    • The Language for rendering Windows Event Logs is now configurable. Up to version 1.4.x The LogScale Collector used the system language to render the event message, collected as @rawstring. This has the potential downside, that for fleets with Windows hosts using different system languages, the collected @rawstring will differ. This only applies for rendering of the event message (no other values) and only for local events.

      In the case of forwarded events the message is rendered locally by the Windows Event Forwarded, and when collected on the Windows Event Forwarder, the message is sent as plain text to the LogScale Collector.

      A new config parameter (language) for setting the render language using Windows LCID codes has been added. The default setting is 0, which corresponds to the previous behaviour, which is the active language on the host.

Bug Fixes

  • Collecting Data

    • Misalignment of field names for the Windows event log source has been corrected, see above.

Falcon Log Collector 1.4.0 GA (2023-5-08)
Version?Type?Release Date?

Config.

Changes?
1.4.0GA2023-5-08no

Hide file hashes

Show file hashes

Bugfix for the Windows event log source, improvements to fleet Management.

Improvements, new features and functionality

  • Other

    • Checkpointer has been improved

      In preparation for future improvements, the checkpoint database has been changed from a JSON file to a binary database format. The existing checkpoints.json file will be automatically imported into the new database. The LogScale Collector will now write a backwards compatible checkpoints.json file on shutdown, which will not be re-imported.

    • Command line arguments

      The LogScale Collector command line interface has been changed to use -- (double dash) for each option. Existing - (single dash) options will be converted in a transition period. A deprecation warning is emitted when options are provided with only a single dash.

  • Fleet Management

    • Fleet Management Improvements

      • When enrolling a LogScale Collector to Fleet management, the enroll process will now stop and start the service during the enrollment process. This behaviour can be omitted by using the flag to the humio-log-collector enroll command.

      • After a successful enrollment, the LogScale Collector service will be configured to automatically start after a reboot. This behaviour can be omitted by using the --no-service flag to the humio-log-collector enroll command.

      • The LogScale Collector process will now exit if it receives an 401 Unauthorized error code during a Fleet management poll operation. The error code signals that the instance no longer has access to the LogScale cluster and cannot be managed. The service manager will automatically restart the LogScale Collector after exiting.

      • When enrolled in Fleet Management, the LogScale Collector will now collect diagnostics from the sinks and send them to Fleet management. The diagnostics will contain various warning and error states that might occur when sending events to LogScale. The diagnostics is available for viewing in the Fleet management tab in LogScale.

Bug Fixes

  • Managing Data

    • Corrected the handling of subscription to more than 64 channels in a single Windows event log source.

      The wineventlog source sometimes encountered issues when configured with more than 64 channels in a single Windows event log source (type: wineventlog). In this scenario it would not collect any events, and the following error message was observed: "extNext: The operation identifier is not valid.". .

Falcon Log Collector 1.3.4 GA (2023-3-30)
Version?Type?Release Date?

Config.

Changes?
1.3.4GA2023-3-30no

Hide file hashes

Show file hashes

Bugfix for the Windows event log source, related to an issue with forwarded events.

Bug Fixes

  • Collecting Data

    • Using the enroll command, to enroll a new collector to fleet management in a linux environment,would previously cause an error if the collector had not been running before, i.e. if the enroll command is the first action.

      When enrolling a new collector, the collector would use an empty machine id value due to incorrect permissions set up by the enroll command. This is not a problem when enrolling collectors that have already been run.

      Starting with this release the enroll command no longer has this issue. In case the above error is encountered, a manual fix is required to give the service user the correct permissions: 

      sudo chown humio-log-collector:humio-log-collector /var/lib/humio-log-collector/.machine-id

    • In a setup using the Windows event log source for collecting forwarded events, the collector has been seen to crash while parsing forwarded events.

      This may occur in a scenario where the remote WEF (Windows Event Forwarding client) and the WEC (Windows Event Collector) go online after a restart. The re-initiated event subscription causes an exception, which stops the collector. This has now been corrected.

Known Issues

  • Collecting Data

    • When collecting data from a Windows event, the collector extracts information from event data and maps the data to named fields in LogScale.

      In scenarios with forwarded events containing empty data values, the indexing of values and names can become misaligned. In this case the current parsing approach is not possible due to misalignment of field names and values. Previously this would result in incorrect values being assigned to field names.

      Starting with this release the Collector appends these values as indexed fields (windows.EventData[0..n]) instead of named fields, and introduces a new field @collect.error with the value: "wineventlog: couldn't parse names for event data".

Falcon Log Collector 1.3.3 Withdrawn (2023-3-21)
Version?Type?Release Date?

Config.

Changes?
1.3.3Withdrawn2023-3-21no

Hide file hashes

Show file hashes

This release has been withdrawn due to the introduction of a regression which could result in missing @rawstring for the Windows event log source.

If you are using this version we recommend you upgrade to 1.3.4.

Falcon Log Collector 1.3.2 GA (2023-3-16)
Version?Type?Release Date?

Config.

Changes?
1.3.2GA2023-3-16No

Hide file hashes

Show file hashes

Bugfix for the Fleet Management communication, eliminating excessive retries.

Bug Fixes

  • Fleet Management

    • If Fleet Management communication with LogScale is unsuccessful the LogScale Collector will do exponential backoff.

      In some scenarios, an error in the backoff implementation caused the retry timeout to drop to zero, resulting in excessive retries. This is now corrected.

Falcon Log Collector 1.3.1 GA (2023-3-9)
Version?Type?Release Date?

Config.

Changes?
1.3.1GA2023-3-9Yes

Hide file hashes

Show file hashes

Bugfix for the Windows event log source, related to an issue with the event data fields.

Improvements, new features and functionality

  • Configuration

    • When installing on Linux the provided service file allowing to run the collector as a systemd service, now defaults to "Restart=always". This is to ensure that unless the service is stopped, the collector service will always be restarted in case of e.g. a crash.

    • The behaviour in cases where the system HTTP proxy detection fails, has been changed.

      If no proxy is configured, the collector will attempt to detect and use the system HTTP proxy. Previously if detection failed the collector would stop, for example this sometimes occurred on older versions of Windows.

      Now in case of failure a warning will be logged, and the collector will continue without a proxy (corresponding to the configuration: proxy:none).

  • Debugging

    • Usability improvement of the enroll command.

      The check for supplied command line arguments is improved and if incorrect/missing arguments are encountered usage is printed.

Bug Fixes

  • Collecting Data

    • Corrected handling of event templates version for the Windows event log source (type: wineventlog).

      When collecting data from a Windows Event, the collector extracts information from event data and maps the data to named fields in LogScale.

      Scenarios where an event has multiple versions of its XML template were not handled correctly, potentially resulting in incorrect values being assigned to field names.

  • Fleet Management

    • Corrected UserAgent in HTTP requests for fleet overview and fleet management (Internal improvement).

Falcon Log Collector 1.3.0 GA (2023-2-7)
Version?Type?Release Date?

Config.

Changes?
1.3.0GA2023-2-7Yes

Hide file hashes

Show file hashes

Fleet management now supports remote configuration of LogScale Collectors. This gives an administrator the option of managing the configuration of LogScale Collector instances in LogScale, instead of managing configuration files directly on the device where LogScale Collector is installed.

Improvements, new features and functionality

  • Configuration

    • The configuration of LogScale Collectors can be managed in LogScale. This is accomplished using configurations and enrollment tokens stored in LogScale.

      To be able to manage the configuration of collectors in LogScale, collectors need to be enrolled to remote configuration, this is done using enrollment tokens.

      Two new pages have been added to the Fleet Management tab in the LogScale user interface.

      • The Config overview page, lists all available configurations and the number of LogScale Collectors using each configuration. The page furthermore allows you to create new configurations. See Manage Remote Configurations for more information.

      • The Enrollment tokens page lists all available enrollment tokens, and allows for creating new enrollment tokens.

        The actual enrollment of a LogScale Collector is performed by executing an enrollment command on the device with the installed LogScale collector instance. The command to be executed can be grabbed from the enrollment token page. See Manage Falcon Log Collector Instance Enrollment for more information.

      The Fleet overview page, which displays the status of all LogScale Collector instances, now includes the name of the assigned configuration to each LogScale Collector.

      It is still possible to use the Fleet Overview without enrolling LogScale Collector instances in remote configuration, in which case configuration will need to be managed directly on the device with installed collector. See Fleet Management Overview for more information.

Falcon Log Collector 1.2.3 GA (2023-1-23)
Version?Type?Release Date?

Config.

Changes?
1.2.3GA2023-1-23No

Hide file hashes

Show file hashes

This version contains bug fixes.

Bug Fixes

  • Collecting Data

    • Fixed a bug on Windows where the Log Collector locks open log files, preventing applications from rotating log files via rename or delete.

Falcon Log Collector 1.2.2 GA (2023-1-16)
Version?Type?Release Date?

Config.

Changes?
1.2.2GA2023-1-16Yes

Hide file hashes

Show file hashes

Bug fixes, improvements and Windows log format collecting features.

Improvements, new features and functionality

  • Collecting Data

    • Added an option to WinEventLog source for including/excluding the XML.

    • Moved default program data directory on Windows to prevent possible conflicts with Falcon Sensor.

    • Improved performance of the WinEventLog source.

    • Added an option to WinEventLog source for excluding eventIDs.

Bug Fixes

  • Collecting Data

    • Fixed a bug which caused the checkpointer for WinEventLog source to not update all of the configured channels.

Falcon Log Collector 1.2.1 GA (2022-11-10)
Version?Type?Release Date?

Config.

Changes?
1.2.1GA2022-11-10No

Hide file hashes

Show file hashes

Bug fix for an issue related to file source which caused it to stop monitoring files.

Bug Fixes

  • Collecting Data

    • Fixed a bug which could cause the file source to stop monitoring files due to a race condition in file creation, update or deletion scenarios.

Falcon Log Collector 1.2.0 GA (2022-10-27)
Version?Type?Release Date?

Config.

Changes?
1.2.0GA2022-10-27Yes

Hide file hashes

Show file hashes

This version of the humio log collector offers the Fleet Overview functionality, which allows you to monitor the status of log collector instances and the following improvements:

  • Improved configuration file validation

  • Improved error logging

  • Reload configuration file feature

  • Using environment variables as the sink url

  • The file source now has more include and exclude patterns and uses less resources by waiting for changes to the file

  • the CMD source can now create single multiline events

  • the wineventlog can now filter events by provider and keep bookmarks of its progress

  • Performance improvements

  • improved batch handling

  • Enforces the use of HTTPS.

Improvements, new features and functionality

  • Collecting Data

    • The url option in the sinks part of the configuration can now refer to an environment variable by using the ${ENVVAR} syntax.

    • The wineventlog source can now filter events based on the provider name. Set the option providers to an array of provider names that should be included to enable this feature. This source also keeps a bookmark of its progress in the Windows event log, and resumes from there when the collector is restarted.

    • The cmd source can now create a single multiline event when running in the schedule mode. Set the option consolidateOutput to true to enable this feature.

    • The file source can now have additional include and exclude patterns in the same configuration. Specifically, the options exclude and include can be either a string or an array of strings.

    • Improved batch handling

      • The sinks now have additional configuration options to change the maximum event size maxEventSize (default 1MB) and the maximum batch size maxBatchSize (default: 16 MB). The limits are propagated to the queue, where it replaces the previous maxEventsPerRequest option. The limits are also propagated to all the sources that reference the sink.

      • The memory queue no longer supports configuration of maxEventsPerRequest, it inherits the maximum bytes per request from the sink maxBatchSize.

      • The memory queue no longer waits before flushing a batch that is larger than the maximum batch size.

      • The collector now warns you when a memory queue reaches 50% and 80% of capacity.

      • The collector now sends a warning after 2 retry attempts when sending events to a http sink.

  • Configuration

    • Improved configuration file validation - The collector is now more thorough when validating its configuration file. An example of this is that unknown options in the configuration are invalid and will prevent running the program. Upon detection of an invalid configuration, the collecter will attempt to provide a descriptive error, some examples of this are: 

      error reading config file "my_config.yaml" sources:
        name must consist of only alphanumeric characters or '.', '_'
        and '-'

            error reading config file "my_config.yaml": 
            sources.cmd_uname_scheduled.interval: invalid type string,
            wanted int` 

            error reading config file "my_config.yaml"
  : sources.dummy_logs.sink: missing value for required field`

    • The collector now enforces using https:// for URLs, this can be overridden by adding the -allow-insecure-http command line flag.

    • The collector now reloads the configuration file when it receives a SIGHUP. This does not apply to the logLevel and dataDirectory options. If the new configuration is invalid, the program will stop.

  • Debugging

    • The default log level is now set to warnings, previously only errors were logged by default.

  • Managing Data

    • Improved memory usage of the memory queue component by removing an upfront buffer that caused it to store more events than specified by the maxLimitInMB option.

    • Improved serialization performance in the humiosink leading to lower memory usage and faster serialization of events.

    • If a file monitored by the file source is inactive not written for a configurable period default: 60 seconds, the file descriptor is closed to release system resources, and watched for changes instead. Whenever the file changes, it is re-opened. This is configurable by the inactivityTimeout option in the file source.

  • Fleet Management

    • The collector now supports reporting to the fleet overview of LogScale. Configure the fleetManagement part of the configuration to enable this feature, see Fleet Management (fleetManagement) for more information.

      When the feature is enabled, the collector will periodically send metrics to LogScale, including the OS version, the collector version, how much data is ingested, and a description of the configured log sources.

Bug Fixes

  • Collecting Data

    • Events from the wineventlog source which contain fields of the type hexadecimal integer were presented as a base 10 number, they are now presented as a base 16 number.

Humio Log Collector 1.1.4 GA (2022-10-12)
Version?Type?Release Date?

Config.

Changes?
1.1.4GA2022-10-12No

Hide file hashes

Show file hashes

Fixed a bug with the Windows event log source.

Bug Fixes

  • Collecting Data

    • Fixed a bug which made the log collector stop when it encountered a Windows event that contained a binary property of zero-length.

Humio Log Collector 1.1.3 GA (2022-10-03)
Version?Type?Release Date?

Config.

Changes?
1.1.3GA2022-10-03No

Hide file hashes

Show file hashes

Improved troubleshooting on Windows, improved checkpointing on disk and fix for a bug on the data sink type.

Improvements, new features and functionality

  • Debugging

    • Improved Checkpointing to disk -- In case of failure writing checkpoints to disk, an error will be logged and writing to disk will be retried with exponential backoff for up to 1 second. This avoids a potential race condition, in which an external program (e.g. an anti-virus program) locks a file that is being simultaneously accessed by the Log Collector.

    • Improved troubleshooting On Windows platforms -- the Log Collector will send errors and warnings to the Windows event log.

Bug Fixes

  • Collecting Data

    • When sending data to a configured sink the http-header: Content-Type is now set to application/json.

Humio Log Collector 1.1.2 Not Released (2022-09-29)
Version?Type?Release Date?

Config.

Changes?
1.1.2Not Released2022-09-29No

Important

This release has been withdrawn due to an issue on Windows, where, in certain configurations, it will continuously log the same event.

If you upgraded to this version we recommend you downgrade to 1.1.1. If you have not installed 1.1.2 upgrade directly to 1.1.3 when it is available.

Humio Log Collector 1.1.1 GA (2022-09-19)
Version?Type?Release Date?

Config.

Changes?
1.1.1GA2022-09-19Yes

Hide file hashes

Show file hashes

Fixed issues on Syslog and JournalD data collection and improved the queue.

Improvements, new features and functionality

  • Managing Data

    • Improved the way events are being queued in order to better respect the maximum limit of the queue.

Bug Fixes

  • Collecting Data

    • Fixed an issue with Syslog data where the source would allocate more memory than was needed.

    • Fixed a JournalD source issue where the collector would stop collecting new events after journal files have been rotated.

    • Fixed an issue when using Syslog source where syslog messages were silently discarded.

    • The Syslog source now limits events to 2 KB (configurable via the maxEventSize parameter on the source).

Humio Log Collector 1.1.0 GA (2022-06-25)
Version?Type?Release Date?

Config.

Changes?
1.1.0GA2022-06-25Yes

Hide file hashes

Show file hashes

Extended support and functionalities.

Improvements, new features and functionality

  • Collecting Data

    • Support for Multiline logs

    • JournalD source support

    • Updated cmd source support

    • The log collector supports for reading gzip and bzip2 compressed files by default.

  • Managing Data

    • Filter Windows event log by EventID

    • Disk queue support

    • Transform Static fields

    • The user can use environment variables to configure:

      • ingest tokens

      • the field values in the static field transform

      • the environment for any command run through the cmd source

    • The queue configuration option fullAction: deleteLatest has been removed are set to the default pause.

Humio Log Collector 1.0.2 LTS (2022-05-05)
Version?Type?Release Date?

Config.

Changes?
1.0.2LTS2022-05-05No

Hide file hashes

Show file hashes

These notes include entries from the following previous releases: 1.0.0, 1.0.1

Improvements, new features and functionality

  • Network

    • Offers network compression which defaults to ON.

    • Supports HTTP(S) proxies.

  • Managing Data

    • Ships all existing events in the file.

    • @collect.* metadata attached to the events including unique collector ID, hostname, @collect.timestamp etc

    • Only handles single line events

    • Collects from local files using a glob pattern (so single file, directory, recursive, etc) and from windows eventlogs and system logs.

    • Buffers in memory.

    • Tails for new events in the file.

    • Offers a sub-second ingest lag between a line being written and sent to Humio (configurable)

Bug Fixes

  • Other

    • Automatically reload the systemd daemon after install on Linux.

    • Fixed a bug where the log collector would get stuck when encountering a long line (131,072 B) and use 100 % CPU.

    • Fixed a bug that caused the log collector to start from the beginning of all files after being restarted.

Humio Log Collector 1.0.1 LTS (2022-04-25)
Version?Type?Release Date?

Config.

Changes?
1.0.1LTS2022-04-25No

Hide file hashes

Show file hashes

These notes include entries from the following previous releases: 1.0.0

Improvements, new features and functionality

  • Network

    • Offers network compression which defaults to ON.

    • Supports HTTP(S) proxies.

  • Managing Data

    • Ships all existing events in the file.

    • @collect.* metadata attached to the events including unique collector ID, hostname, @collect.timestamp etc

    • Only handles single line events

    • Collects from local files using a glob pattern (so single file, directory, recursive, etc) and from windows eventlogs and system logs.

    • Buffers in memory.

    • Tails for new events in the file.

    • Offers a sub-second ingest lag between a line being written and sent to Humio (configurable)

Bug Fixes

  • Other

    • Fixed a bug where the log collector would get stuck when encountering a long line (131,072 B) and use 100 % CPU.

Humio Log Collector 1.0.0 LTS (2022-04-23)
Version?Type?Release Date?

Config.

Changes?
1.0.0LTS2022-04-23Yes

Hide file hashes

Show file hashes

The first release of Humio Log Collector our native Log shipper which can be used to ship local files to a Humio repository by specifying an ingest token. This version of the log collector offers the following features.

Improvements, new features and functionality

  • Network

    • Offers network compression which defaults to ON.

    • Supports HTTP(S) proxies.

  • Managing Data

    • Ships all existing events in the file.

    • @collect.* metadata attached to the events including unique collector ID, hostname, @collect.timestamp etc

    • Only handles single line events

    • Collects from local files using a glob pattern (so single file, directory, recursive, etc) and from windows eventlogs and system logs.

    • Buffers in memory.

    • Tails for new events in the file.

    • Offers a sub-second ingest lag between a line being written and sent to Humio (configurable)