Linux Log Format

Getting system logs and host metrics from your Linux system into Humio allows you to monitor a large array of issues. You can
Find servers that have too much load
Detect when you are running out of disk space
See when services reboot or crash
See which user runs which commands with sudo
Host Metrics
To get standard host metrics, like CPU usage, load, and memory, use Metricbeat. For Linux host metrics, use the system module.
Note
Metricbeat can provide a lot of metrics (many per process for example). Experiment with finding the level you need.
Example Metricbeat Configuration
metricbeat.modules:
- module: system
enabled: true
period: 10s
metricsets:
- cpu
- load
- filesystem
- fsstat
- memory
- network
output.elasticsearch:
hosts: ["https://$$YOUR_LOGSCALE_URL:443/api/v1/ingest/elastic-bulk"]
username: $INGEST_TOKEN
Where $INGEST_TOKEN
is the
ingest token for your repository, (a string such as
fS6Kdlb0clqe0UwPcc4slvNFP3Qn1COzG9DEVLw7v0Ii
).
See Metricbeat for more information.
Note
Example queries: You may be interested in these queries on host metrics.
System Logs (syslog)
To ship interesting system logs from /var/log/
to
Humio use
Filebeat.
Example Filebeat configuration
filebeat.inputs:
- paths:
- /var/log/syslog
- /var/log/auth.log
fields:
"@type": syslog-utc
output.elasticsearch:
hosts: ["https://$YOUR_LOGSCALE_URL:443/api/v1/ingest/elastic-bulk"]
username: $INGEST_TOKEN
Where $INGEST_TOKEN
is the
ingest token for your repository, (a string such as
fS6Kdlb0clqe0UwPcc4slvNFP3Qn1COzG9DEVLw7v0Ii
).
Notice the type is syslog-utc
,
which points to the built-in syslog parser, expecting the timestamp to
be in UTC time. Often syslog timestamps are in local time. Create a
new parser with another timezone in Humio if necessary. You can copy
the built-in syslog-utc and change the timezone. See
Parsing for details.
Check out the Filebeat page for more information.
Custom Logs or Metrics
If you have custom logs or metrics you want to ship we suggest one of these strategies:
Append the logs/metrics to a log file and use Filebeat to ship them similarly to the system logs above.
Use
cron
to run a script that sends data to Humio through the Ingest API.