Linux Log Format

Getting system logs and host metrics from your Linux system into Humio allows you to monitor a large array of issues. You can

  • Find servers that have too much load

  • Detect when you are running out of disk space

  • See when services reboot or crash

  • See which user runs which commands with sudo

Host Metrics

To get standard host metrics, like CPU usage, load, and memory, use Metricbeat. For Linux host metrics, use the system module.

Note

Metricbeat can provide a lot of metrics (many per process for example). Experiment with finding the level you need.

Example Metricbeat Configuration

logscale
metricbeat.modules:
  - module: system
    enabled: true
    period: 10s
    metricsets:
      - cpu
      - load
      - filesystem
      - fsstat
      - memory
      - network

output.elasticsearch:
  hosts: ["https://$$YOUR_LOGSCALE_URL:443/api/v1/ingest/elastic-bulk"]
  username: $INGEST_TOKEN

Where $INGEST_TOKEN is the ingest token for your repository, (a string such as fS6Kdlb0clqe0UwPcc4slvNFP3Qn1COzG9DEVLw7v0Ii).

See Metricbeat for more information.

Note

Example queries: You may be interested in these queries on host metrics.

System Logs (syslog)

To ship interesting system logs from /var/log/ to Humio use Filebeat.

Example Filebeat configuration

logscale
filebeat.inputs:
- paths:
    - /var/log/syslog
    - /var/log/auth.log
  fields:
    "@type": syslog-utc

output.elasticsearch:
  hosts: ["https://$YOUR_LOGSCALE_URL:443/api/v1/ingest/elastic-bulk"]
  username: $INGEST_TOKEN

Where $INGEST_TOKEN is the ingest token for your repository, (a string such as fS6Kdlb0clqe0UwPcc4slvNFP3Qn1COzG9DEVLw7v0Ii).

Notice the type is syslog-utc, which points to the built-in syslog parser, expecting the timestamp to be in UTC time. Often syslog timestamps are in local time. Create a new parser with another timezone in Humio if necessary. You can copy the built-in syslog-utc and change the timezone. See Parsing for details.

Check out the Filebeat page for more information.

Custom Logs or Metrics

If you have custom logs or metrics you want to ship we suggest one of these strategies:

  • Append the logs/metrics to a log file and use Filebeat to ship them similarly to the system logs above.

  • Use cron to run a script that sends data to Humio through the Ingest API.