Zeek (Bro) Network Security Monitor

LogScale can analyze Zeek data. This page describes how to get Zeek data into LogScale Pre-made dashboards for Zeek data can be found further down this page on Zeek Dashboards.

Configure Zeek

First, let's set up Zeek to write logs in JSON. That will make it easier to send them to LogScale.

Seth from Corelight has made a nice Zeek script to support streaming Zeek logs as JSON.

The script requires Zeek/Bro 2.5.2+ Download it here

One way to install the script is to put it in the <bro-directory>/site/ folder and then add the Zeek script to the end of local.bro like this:

@load corelight-logs.bro

The script will add new JSON log files in the Zeek log directory next to the standard CSV log files. The new JSON files will be prepended with corelight_ and otherwise have the same name as its corresponding CSV file. So there will be a corelight_conn.log log file corresponding to the conn.log CSV log file.

By default each JSON log file is rotated every 15 minutes, and four versions of the file are kept. These files will be monitored by Filebeat and data send to LogScale as is described below in the section Configure Filebeat.

Some available configurations options for the Zeek script are:

redef CorelightLogs::disable_default_logs = F;   ## Disable default logs and only log in JSON
redef CorelightLogs::extra_files = 4;            ## number of files to keep when rotating
redef CorelightLogs::rotation_interval = 15mins; ## time before rotating a file

These options can be appended to local.bro

It is also possible to test the script by running:

bro -i eth0 <bro-directory-full-path>/site/json-logs-by-corelight.bro

On Mac the default network interface is en0.

You can follow the above or add the Zeek script in a way matching your installation. With the script in place, and after a restart, Zeek should be logging in JSON format, formatted as JSON objects separated by newlines. Verify this by looking in one of the log files, for example corelight_conn.log.

Configure LogScale

We assume you already have a local LogScale running or is using LogScale as a Service. Look at the Installing LogScale documentation for instructions on how to install LogScale.

If you don't have a repository , create one by clicking Add Repository on the front page of LogScale.

Configure Filebeat

We will use Filebeat to ship Zeek logs to LogScale. Filebeat is a light weight, open source agent that can monitor log files and send data to servers like LogScale. Filebeat must be installed on the server having the Zeek logs. Follow the Filebeat Installation instructions to download and install Filebeat. Then return here to configure Filebeat.

Below is a filebeat.yml configuration file for sending Zeek logs to LogScale:

- paths:
    - "${ZEEK_LOG_DIR}/corelight_*.log" #The file path should be a glob matching the json log files
    type: bro-json
  events: 6000
  flush.min_events: 1000
  flush.timeout: 1s
#-------------------------- ElasticSearch output ------------------------------
  hosts: ["http://$YOUR_LOGSCALE_URL/api/v1/ingest/elastic-bulk"]
  password: "${INGEST_TOKEN}"
  compression_level: 5
  bulk_max_size: 1000
  worker: 3
#================================ Logging =====================================
# Sets log level. The default log level is info.
# Available log levels are: critical, error, warning, info, debug
logging.level: info
logging.selectors: ["*"]

The configuration file has these parameters:

You can replace the parameters in the file or set them as ENV parameters when starting Filebeat. To create an ingest token, follow the instructions on the Ingest Tokens documentation page.

Note that in the filebeat configuration we specify that LogScale should use the built-in parser bro-json to parse the data with:

type: bro-json

As Zeek often generates a lot of data we have configured Filebeat to use 3 workers, a bulk_max_size of 1000 and then configured the in memory queue queue.mem accordingly. Experiment with increasing this if filebeat cannot keep up with sending data.

Run Filebeat

With the configuration in place we are ready to run Filebeat.

Run Filebeat as per Running Filebeat instructions. An example of running Filebeat with the above parameters as environment variables:

ZEEK_LOG_DIR=/home/bro/logs HOST=localhost INGEST_TOKEN=******************** /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml


Logging is Verbose: Logging is set to debug in the above Filebeat configuration. It can be a good idea to set it to info when things are running well. Filebeat log files are by default rotated and only 7 files of 10 megabytes each are kept, so it should not fill up the disk. See more in Elastic's Documentation,

If there is data in the Zeek log files, Filebeat will start shipping the data to LogScale. Go to the zeek repository in LogScale and data should be streaming in. Filebeat starts shipping data from the start of the file. If data is old, widen the default search interval in LogScale. To see data flowing into LogScale in realtime, select a time interval of "1m window". This will "tail" the data as it arrives in LogScale.

Zeek Dashboards

Corelight has created some nice Zeek dashboards, that can be downloaded as a package from the LogScale Package Marketplace. The package is named corelight/sensor.

For more Information, see Zeek, Corelight, and LogScale help make observability accessible.