Falcon LogScale Collector 1.7.4 GA (2024-10-03)

Version?Type?Release Date?Config.Changes?
1.7.4GA2024-10-03no

Hide file hashes

Show file hashes

Performance improvements for file sources and syslog sources.

Improvements, new features and functionality

  • Collecting Data

    • File source optimization for Windows and macOS.

      The filesource keeps monitoring files after all data has been read and ingested to be able to continue shipping when/if new data is added. In scenarios with a high number of files, this can be rather CPU consuming on Windows and macOS. In order to reduce the CPU usage a dynamic file scanner, which balances the CPU usage of the file scanning part has been introduced.

  • Other

    • Symbol names and debug information are now stripped from binaries, this results in smaller distributables.

  • Debugging

    • Internal log messages in the Log Collector have been improved. An internal buffer has been increased to avoid missing internal logs and log messages for the syslog source have been augmented with more detail.

  • Fleet Management

    • The Log Collector now supports sending custom labels to Fleet Management.

      This is in preparation for an upcoming Fleet Management feature which allows using labels for defining collector groups. When creating a group - labels can be used in the filter query. For example, labels.myLabel=foo. Labels must be added to the local fleet management config file of the Log Collector. Label values can be expanded from environment variables as well.

  • Installation and Deployment

    • The syslog source has been optimized to use less memory in setups with high amounts of short lived TCP connections. The new approach utilizes a memory pool instead of allocating a new memory for each connection.

      The number of concurrent TCP connections is limited to 1024. The default MaxEventSize for syslog over TCP is changed to 1 MB to match the same setting when using syslog over TLS.

      The previous default setting was 2048 B (which is mentioned in the RFC), however as some users have experienced truncated events, the setting has been changed.

Bug Fixes

  • Other

    • We have identified a previously unhandled scenario in which the Log Collector attempts to send data to a Data Connector and either a HTTP intermediary, such as a proxy, or the Data Connector accepts the HTTP connection, however never returns a HTTP response and in the same time keeps the connection alive.

      Previously this would cause the Log Collector to wait for the response, thus blocking further data on that sink. To address this scenario, the Log Collector will now timeout, and attempt re-transmission, if it does not receive a response within 60 s. In this case a warning: "timeout awaiting response headers" will be logged.

  • Debugging

    • A race condition related to file rotation using compression could cause the checkpointer to get in a state where it would repeatedly log the following error messages, "File failed, waiting 1min. error: EOF" and "pipeline failed, error: EOF".

      The Log Collector now marks the checkpoint for the file as done and the warning message "Handling unexpected EOF in compressed file" will be logged once.