Troubleshooting: Beats Fails to Send Logs due to Filename Issues

Last Updated: 2021-08-20

Affects:

  • Filebeat™ version(s) all

  • Winlogbeat™ version(s) all

  • Metricbeat™ version(s) all

  • Packetbeat™ version(s) all

Condition or Error

Filebeat or Winlogbeat fails to send logs on Windows

Filebeat runs without errors but no data reaches Humio

New files have been added to the configuration but the data does not reach Humio

When using one of the Beats log shippers to load data into Humio, data fails to load and there is no error within the logs.

Causes

  • Some errors related to the parsing and sending of files within Beats are caused by issues within the Beats configuration file. The following factors should be checked:

    Check the filename within the configuration within the paths section points to the correct file:

    ini files
    filebeat.inputs:
     - paths:
     - /var/log/fake-system.log
     encoding: utf-8
  • If the file contains special characters, such as spaces, you can enclose the filename within single quotes to qualify the name. This can be important on systems where spaces or other special characters are regularly used, for example Windows or Mac OS X:

    ini files
    filebeat.inputs:
     - paths:
     - 'C:\apache\logs\httpd.log'
     encoding: utf-8
  • Ensure that the indentation of the file is correct. The file path should be indented under the paths clause. For example:

    ini files
    filebeat.inputs:
     - paths:
     - /var/log/system.log
     encoding: utf-8

    If you have specified multiple files, each file should be indented at the same level:

    ini files
    filebeat.inputs:
     - paths:
     - /apache/log/httpd.log
     - /nodejs/log/httpd.log
     encoding: utf-8
  • Explicitly add an enabled clause within the filename log configuration:

    ini files
    filebeat.inputs:
     - paths:
     - /apache/log/httpd.log
     enabled: true
     encoding: utf-8

Solutions

  • Ensuring that the configuration file is correctly configured. A full sample configuration file is shown below:

    ini files
    filebeat.inputs:
     - paths:
     - /var/log/system.log
     encoding: utf-8
    
     queue.mem:
     events: 8000
     flush.min_events: 1000
     flush.timeout: 1s
    
     output:
     elasticsearch:
     # Using the standard Humio API (preferred)
     hosts: ["example.com:443/api/v1/ingest/elastic-bulk"]
     username: anything
     password: c910f252-8c7a-487d-87ec-1df0d2b6d18a
     compression_level: 5
     bulk_max_size: 200
     worker: 5
    
    
     logging:
     level: debug
     to_files: true
     to_syslog: false
     files:
     path: ./filebeat-logs/
     name: filebeat.log
     keepfiles: 3