Zeek (Bro) Network Security Monitor

Humio is an excellent tool for analyzing Zeek data. This document describes how to get Zeek data into Humio Pre-made dashboards for Zeek data can be found further down this page on Zeek Dashboards.
Configure Zeek
First, let's set up Zeek to write logs in JSON. That will make it easier to send them to Humio.
Seth from Corelight has made a nice Zeek script to support streaming Zeek logs as JSON.
The script requires Zeek/Bro 2.5.2+ Download it here
One way to install the script is to put it in the
<bro-directory>/site/
folder and then add the Zeek script to the end of
local.bro
like this:
@load corelight-logs.bro
The script will add new JSON log files in the Zeek log directory next
to the standard CSV log files. The new JSON files will be prepended
with corelight_
and otherwise
have the same name as its corresponding CSV file. So there will be a
corelight_conn.log
log file
corresponding to the conn.log
CSV log file.
By default each JSON log file is rotated every 15 minutes, and four versions of the file are kept. These files will be monitored by Filebeat and data send to Humio as is described below in the section Configure Filebeat.
Some available configurations options for the Zeek script are:
redef CorelightLogs::disable_default_logs = F; ## Disable default logs and only log in JSON
redef CorelightLogs::extra_files = 4; ## number of files to keep when rotating
redef CorelightLogs::rotation_interval = 15mins; ## time before rotating a file
These options can be appended to
local.bro
It is also possible to test the script by running:
bro -i eth0 <bro-directory-full-path>/site/json-logs-by-corelight.bro
On Mac the default network interface is
en0
.
You can follow the above or add the Zeek script in a way matching your
installation. With the script in place, and after a restart, Zeek
should be logging in JSON format, formatted as JSON objects separated
by newlines. Verify this by looking in one of the log files, for
example corelight_conn.log
.
Configure Humio
We assume you already have a local Humio running or is using Humio as a Service. Look at the Installing LogScale On Your Infrastructure documentation for instructions on how to install Humio.
If you don't have a repository , create one by clicking Add Repository on the front page of Humio.
Configure Filebeat
We will use Filebeat to ship Zeek logs to Humio. Filebeat is a light weight, open source agent that can monitor log files and send data to servers like Humio. Filebeat must be installed on the server having the Zeek logs. Follow the Filebeat Installation instructions to download and install Filebeat. Then return here to configure Filebeat.
Below is a filebeat.yml
configuration file for sending Zeek logs to Humio:
filebeat.inputs:
- paths:
- "${ZEEK_LOG_DIR}/corelight_*.log" #The file path should be a glob matching the json log files
fields:
type: bro-json
queue.mem:
events: 6000
flush.min_events: 1000
flush.timeout: 1s
#-------------------------- ElasticSearch output ------------------------------
output.elasticsearch:
hosts: ["http://$YOUR_LOGSCALE_URL/api/v1/ingest/elastic-bulk"]
password: "${INGEST_TOKEN}"
compression_level: 5
bulk_max_size: 1000
worker: 3
#================================ Logging =====================================
# Sets log level. The default log level is info.
# Available log levels are: critical, error, warning, info, debug
logging.level: info
logging.selectors: ["*"]
The configuration file has these parameters: -
$ZEEK_LOG_DIR
-
$YOUR_LOGSCALE_URL
-
$INGEST_TOKEN
You can replace the parameters in the file or set them as ENV parameters when starting Filebeat. To create an ingest token, follow the instructions on the Ingest Tokens documentation page.
Note that in the filebeat configuration we specify that Humio should
use the built-in parser bro-json
to parse the data with:
fields:
type: bro-json
As Zeek often generates a lot of data we have configured Filebeat to
use 3 workers
, a
bulk_max_size
of 1000 and then
configured the in memory queue
queue.mem
accordingly.
Experiment with increasing this if filebeat cannot keep up with
sending data.
Run Filebeat
With the configuration in place we are ready to run Filebeat.
Run Filebeat as per Running Filebeat instructions. An example of running Filebeat with the above parameters as environment variables:
ZEEK_LOG_DIR=/home/bro/logs HOST=localhost INGEST_TOKEN=******************** /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml
Note
Logging is Verbose: Logging is set to debug in the above Filebeat configuration. It can be a good idea to set it to info when things are running well. Filebeat log files are by default rotated and only 7 files of 10 megabytes each are kept, so it should not fill up the disk. See more in Elastic's Documentation,
If there is data in the Zeek log files, Filebeat will start shipping the data to Humio. Go to the zeek repository in Humio and data should be streaming in. Filebeat starts shipping data from the start of the file. If data is old, widen the default search interval in Humio. To see data flowing into Humio in realtime, select a time interval of "1m window". This will "tail" the data as it arrives in Humio.
Search Zeek Data
With everything in place, Zeek data is streaming into Humio. In the
above Filebeat configuration events are given a
#path tag describing from which file they
originate. To search for data from the
http.log
:
#path=http
Or search data from the conn.log
#path=conn
Leave out the #path filter to search across all files. For example we could count how many events we have in the different files:
groupBy(#path, function=count())
Or show the event distribution over time
timechart(#path, unit="1/minute")
If you are new to Humio and its search capabilities, read the Getting Started tutorial. There is a link to the tutorial in the top right corner of the Humio UI.
Zeek Dashboards
Corelight has created some nice Zeek dashboards, that can be
downloaded as a package from the
Humio Package
Marketplace. The package is named
corelight/sensor
.
For more Information, see Zeek, Corelight, and Humio help make observability accessible.