Docker Log Format

In this guide, we assume that you use Docker in the standard way, where logs are captured from stdout and stderr. If you're looking for how to run LogScale in a Docker container, see the Installing Using Containers instead.

LogScale has full support for the Docker Splunk logging driver. Getting logs from a Docker container is as simple as setting the logging driver and adding the splunk-url and splunk-token logging options to the container

shell
$ docker run --rm -it \
  --log-driver=splunk \
  --log-opt splunk-url=$YOUR_LOGSCALE_URL \
  --log-opt splunk-token=$INGEST_TOKEN \
  alpine ping 8.8.8.8

The $YOUR_LOGSCALE_URL variable is the base URL of your LogScale server, either LogScale Cloud or self-hosted. The $INGEST_TOKEN is the ingest token for your repository.

Parsing Logs

Since Docker handles log lines from stdout as text blobs, you must parse the lines to get the full value from them. To do this, you can either use a built-in parser, or create new ones for your log types. For more details on creating parsers, see Parsers.

In terms of log management, Docker is a transport layer. Before writing a custom parser, see Built-in Parsers to see if LogScale already supports your log type.

Configuring Docker Daemon

To configure the Docker daemon to forward all logs for all containers by default you'll have to update the daemon.json configuration file with the following parameters:

javascript
{
  "log-driver" : "splunk",
  "log-opts" : {
    "splunk-token" : "$INGEST_TOKEN",
    "splunk-url" : "$YOUR_LOGSCALE_URL"
  }
}

When finished, restart the Docker daemon.

To exclude from log forwarding, you can run your container with the default json-file logging driver

shell
$ docker run --log-driver=json-file --rm alpine whoami

By default, Docker logging drivers are blocking, meaning that they will prevent the process from printing to stdout and stderr while logs are being handled. This can, and should be, controlled by the mode log-opt.

In addition to the mode, the Splunk logging driver has it's own buffer, which will postpone the process pausing somewhat. Also, Docker will discard the oldest logs in non-blocking mode when the buffer is full.

Docker Daemon Metrics

To get standard host level metrics for your docker containers, use Metricbeat. It includes a docker module.

Below is an example configuration of Metricbeat:

yaml
metricbeat.modules:
  - module: docker
    metricsets: ["cpu", "info", "memory", "network", "diskio", "container"]
    hosts: ["unix:///var/run/docker.sock"]
    enabled: true
    period: 10s

output.elasticsearch:
  hosts: ["$YOUR_LOGSCALE_URL/api/v1/ingest/elastic-bulk"]
  username: my-organization
  password: $INGEST_TOKEN

Where:

See also Elastic Beats for more information.