The humio-fleet Repository

The humio-fleet repository is a system repository that stores metadata from log shippers, used by the Fleet Management UI.

LogScale Collector Metrics

The LogScale Collector sends metrics to Fleet Management for enrolled instances of LogScale Collector. The LogScale Collector reports the following metrics, which could be used to create dashboards to monitor your fleet:

  • Process CPU Usage: The CPU usage (%) of the LogScale Collector process.

  • Process Memory Usage: The memory usage of the LogScale Collector process.

  • Host Disk Usage: The full percentage of the disk partition that the process is using as the data directory. The metrics are automatically ingested into the humio-fleet system repository.

Prerequisites

The following prerequisites must be met to use this feature:

  • Running LogScale Collector newer or equal to 1.5.1.

  • Running LogScale newer or equal to 1.100.

  • The LogScale Collector has been enrolled using the enrollment command.

LogScale Collector Metrics

The LogScale Collector sends the following structured JSON events to the humio-fleet repository. The events contain a discriminator field called kind which can be used to determine what kind of metric is being reported.

The kindfield is being converted to a LogScale tag, as such the field name is converted to #kind, but the key in the JSON event is called kind. This is relevant when querying the data in humio-fleet, since it must be filtered like: #kind=system.

The LogScale Collector also sends an event that doesn't contain the #kind tag. Those events are deprecated, but are currently being used to generate the Fleet Overview page. Use the following descriptions to understand the JSON events that are being stored in the humio-fleet repository.

Kind system

The following is an example of a system event: 

{
  "kind": "system",
  "id": "gmIhP974udWeeaMzRKPOrxa557OfGTMp",
  "ipAddress": "198.51.100.1",
  "timestamp": 1690840800000,
  "version": "1.5.0",
  "hostname": "logscale-collector-example",
  "system": "Ubuntu 20.04.6 LTS (Focal Fossa) (amd64)",
  "startTime": 1690840800000,
  "bootTime": 1690840800000,
  "mode": "full",
  "machineId": "c53aa5a4-2828-475f-836a-97c1a933b0b5"
}
Field Type Required Description
kind constant system yes The kind field determines the type of event.
id string yes The globally unique ID assigned to the enrolled LogScale Collector. Used to correlate the metrics events.
ipAddress string yes The source IP address of the LogScale Collector as seen by Fleet Management.
timestamp number yes The timestamp of the event in epoch milliseconds.
version string yes The version of the LogScale Collector.
hostname string yes The system hostname of the LogScale Collector.
system string yes A descriptive string of the operating system that is running the LogScale Collector.
startTime number yes The start time of the LogScale Collector process in epoch milliseconds.
bootTime number yes The operating system boot time in epoch milliseconds.
mode string yes The current mode of Fleet Management. The value is full when using remote configuration.
machineId string yes The machine ID generated by the LogScale Collector locally when it was first installed.

Kind collectorCpuUsage

The following is an example of a collectorCpuUsage event: 

{
  "kind": "collectorCpuUsage",
  "id": "gmIhP974udWeeaMzRKPOrxa557OfGTMp",
  "bucketStart": 1690840800000,
  "bucketEnd": 1690840860000,
  "avg": 4.12,
  "cores": 8
}
Field Type Required Description
kind constant collectorCpuUsage yes The kind field determines the type of event.
id string yes The globally unique ID assigned to the enrolled LogScale Collector. Used to correlate the metrics events.
bucketStart number yes The bucket start timestamp in epoch milliseconds. The value of the event covers the interval [bucketStart;bucketEnd].
bucketEnd number yes The bucket end timestamp in epoch milliseconds. The value of the event covers the interval [bucketStart;bucketEnd].
avg number yes The CPU usage of the LogScale Collector process averaged over the bucket duration. A value of 100 % corresponds to one vCPU utilized completely.
cores number yes The number of CPU cores in the system. Dividing avg by cores results in a value between 0 % and 100 %.

Kind collectorMemoryUsage

The following is an example of a collectorMemoryUsage event: 

{
  "kind": "collectorMemoryUsage",
  "id": "gmIhP974udWeeaMzRKPOrxa557OfGTMp",
  "bucketStart": 1690840800000,
  "bucketEnd": 1690840860000,
  "max": 34848768
}
Field Type Required Description
kind constant collectorMemoryUsage yes The kind field determines the type of event.
id string yes The globally unique ID assigned to the enrolled LogScale Collector. Used to correlate the metrics events.
bucketStart number yes The bucket start timestamp in epoch milliseconds. The value of the event covers the interval [bucketStart;bucketEnd].
bucketEnd number yes The bucket end timestamp in epoch milliseconds. The value of the event covers the interval [bucketStart;bucketEnd].
max number yes The memory usage of the LogScale Collector process in bytes. The value is the maximum value over the bucket duration.

Kind hostDiskUsage

The following is an example of a hostDiskUsage event: 

{
  "kind": "hostDiskUsage",
  "id": "gmIhP974udWeeaMzRKPOrxa557OfGTMp",
  "bucketStart": 1690840800000,
  "bucketEnd": 1690840860000,
  "path": "/var/lib/logscale-collector",
  "max": 8.62
}
Field Type Required Description
kind constant hostDiskUsage yes The kind field determines the type of event.
id string yes The globally unique ID assigned to the enrolled LogScale Collector. Used to correlate the metrics events.
bucketStart number yes The bucket start timestamp in epoch milliseconds. The value of the event covers the interval [bucketStart;bucketEnd].
bucketEnd number yes The bucket end timestamp in epoch milliseconds. The value of the event covers the interval [bucketStart;bucketEnd].
max number yes The disk usage in percent of the partition containing the path path.
path string yes The path that is being monitored. Currently specifies the data directory of the LogScale Collector.

LogScale Collector Metadata

Each event has some metadata attached to it on ingestion; all metadata fields start with @ to make them easy to identify. All events will contain the following metadata fields by default.

Metadata Field Description
@collect.host Name of the ingesting host
@collect.id Unique ID of the collector
@collect.timezone Timezone
@collect.timestamp Timestamp
@collect.source_name Name of the source.
@collect.source_type (e.g. cmd, file, journald, syslog, syslog_tls, unifiedlog, wineventlog)
@collect.error Error occurred while collecting data, e.g. wineventlog: could not parse names for event data.

The following additional metadata fields are source specific.

Source Metadata Field Description
journald @collect.unit Name of the unit, e.g. ntp.service
file @collect.file File name from where the event is collected.
wineventlog @collect.channel Channel of the collected event.
syslog @collect.remote Remote IP address and port.
@collect.socket Local socket e.g. :514/UDP
command @collect.cmd The command which is executed.
@collect.pid The PID of the executed command
@collect.stream The output stream of the executed command, stdout or stderr.