The humio-fleet Repository

The humio-fleet repository is a system repository that stores metadata from log shippers, used by the Fleet Management UI.

Important

Data in system repositories cannot be redacted. For more information about immutability of data, see Immutability of data.

Log Collector Metrics

The Log Collector sends metrics to Fleet Management for enrolled instances of Log Collector. The Log Collector reports the following metrics, which could be used to create dashboards to monitor your fleet:

  • Process CPU Usage: The CPU usage (%) of the Log Collector process.

  • Process Memory Usage: The memory usage of the Log Collector process.

  • Host Disk Usage: The full percentage of the disk partition that the process is using as the data directory. The metrics are automatically ingested into the humio-fleet system repository.

Prerequisites

The following prerequisites must be met to use this feature:

  • Running Log Collector newer or equal to 1.11.2.

  • Running LogScale newer or equal to 1.234.

  • The Log Collector has been enrolled using the enrollment command.

Log Collector Metrics

The Log Collector sends the following structured JSON events to the humio-fleet repository . The events contain a discriminator field called kind which can be used to determine what kind of metric is being reported.

The kind field is being converted to a LogScale tag, as such the field name is converted to #kind, but the key in the JSON event is called kind. This is relevant when querying the data in humio-fleet, since it must be filtered like: #kind=system.

The Log Collector also sends an event that doesn't contain the #kind tag. Those events are deprecated, but are currently being used to generate the Fleet Overview page. Use the following descriptions to understand the JSON events that are being stored in the humio-fleet repository .

(Note that, given the version of the collector, there are no more deprecated events).

Kind system

The following is an example of a system event: 

{
  "kind":"system",
  "id":"2uxpMKt67xO5mqkA71LQ8fhITZjljEZd",
  "ipAddress":"198.51.100.1",
  "timestamp":1777633648388,
  "version":"1.11.2",
  "hostname":"ML-MWQ0GJVDYC",
  "system":"macOS 26.4 (arm64)",
  "startTime":1777633647347,
  "bootTime":1777269972665,
  "mode":"full",
  "machineId":"75257cf7-0f4e-49f6-956d-d3856b4d366c",
  "os":"darwin",
  "userAgent":"logscale-collector-fips-darwin-arm64/debug"
}
Field Type Required Description
allowRemoteCMD boolean no Whether the collector allows remote command execution.
bootTime number yes The operating system boot time in epoch milliseconds.
hostname string yes The system hostname of the Log Collector.
id string yes The globally unique ID assigned to the enrolled Falcon LogScale Collector. Used to correlate the metrics events.
ipAddress string yes The source IP address of the Log Collector as seen by Fleet Management.
kind constant system yes The kind field determines the type of event.
machineId string yes The machine ID generated by the Log Collector locally when it was first installed.
mode string yes The current mode of Fleet Management. The value is full when using remote configuration.
os string yes The operating system on which the collector is running. Possible values are Windows, Linux, macOS, or Unknown.
system string yes A descriptive string of the operating system that is running the Log Collector.
startTime number yes The start time of the Log Collector process in epoch milliseconds.
timestamp number yes The timestamp of the event in epoch milliseconds.
userAgent string yes The user agent of the collector.
version string yes The version of the Log Collector.

Kind collectorCpuUsage

The following is an example of a collectorCpuUsage event: 

{
  "kind": "collectorCpuUsage",
  "id": "gmIhP974udWeeaMzRKPOrxa557OfGTMp",
  "bucketStart": 1690840800000,
  "bucketEnd": 1690840860000,
  "avg": 4.12,
  "cores": 8
}
Field Type Required Description
kind constant collectorCpuUsage yes The kind field determines the type of event.
id string yes The globally unique ID assigned to the enrolled Falcon LogScale Collector. Used to correlate the metrics events.
bucketStart number yes The bucket start timestamp in epoch milliseconds. The value of the event covers the interval [bucketStart;bucketEnd].
bucketEnd number yes The bucket end timestamp in epoch milliseconds. The value of the event covers the interval [bucketStart;bucketEnd].
avg number yes The CPU usage of the Log Collector process averaged over the bucket duration. A value of 100 % corresponds to one vCPU utilized completely.
cores number yes The number of CPU cores in the system. Dividing avg by cores results in a value between 0 % and 100 %.

Kind collectorMemoryUsage

The following is an example of a collectorMemoryUsage event: 

{
  "kind": "collectorMemoryUsage",
  "id": "gmIhP974udWeeaMzRKPOrxa557OfGTMp",
  "bucketStart": 1690840800000,
  "bucketEnd": 1690840860000,
  "max": 34848768
}
Field Type Required Description
kind constant collectorMemoryUsage yes The kind field determines the type of event.
id string yes The globally unique ID assigned to the enrolled Falcon LogScale Collector. Used to correlate the metrics events.
bucketStart number yes The bucket start timestamp in epoch milliseconds. The value of the event covers the interval [bucketStart;bucketEnd].
bucketEnd number yes The bucket end timestamp in epoch milliseconds. The value of the event covers the interval [bucketStart;bucketEnd].
max number yes The memory usage of the Log Collector process in bytes. The value is the maximum value over the bucket duration.

Kind hostDiskUsage

The following is an example of a hostDiskUsage event: 

{
  "kind": "hostDiskUsage",
  "id": "gmIhP974udWeeaMzRKPOrxa557OfGTMp",
  "bucketStart": 1690840800000,
  "bucketEnd": 1690840860000,
  "path": "/var/lib/logscale-collector",
  "max": 8.62
}
Field Type Required Description
kind constant hostDiskUsage yes The kind field determines the type of event.
id string yes The globally unique ID assigned to the enrolled Falcon LogScale Collector. Used to correlate the metrics events.
bucketStart number yes The bucket start timestamp in epoch milliseconds. The value of the event covers the interval [bucketStart;bucketEnd].
bucketEnd number yes The bucket end timestamp in epoch milliseconds. The value of the event covers the interval [bucketStart;bucketEnd].
max number yes The disk usage in percent of the partition containing the path path.
path string yes The path that is being monitored. Currently specifies the data directory of the Log Collector.

Kind sources

The following is an example of a sources event: 

{
  "kind":"sources",
  "id":"XWt9NuWvgPpjhMQmtXAoQ6NCoAFRddpr",
  "timestamp":1777633438430,
  "sources":[
  {
  "collectedBatches":20,
  "collectedBytes":76391,
  "collectedEvents":194,
  "errors":[],
  "repository": "humio"
  "parser":"apple/unifiedlog:unifiedlog-compact",
  "sinkType":"humio",
  "sourceName":"macoslog",
  "sourceType":"unifiedlog"
  }]
}
Field Type Required Description
kind constant sources yes The kind field determines the type of event.
id string yes The globally unique ID assigned to the enrolled Falcon LogScale Collector. Used to correlate the metrics events.
timestamp integer yes The timestamp of the event in epoch milliseconds.
sources array yes Array of source entries.

Each entry in sources:

Field Type Required Description
sourceName string yes Name of the source.
sourceType string no Type of source.
sinkType string yes Type of sink receiving data.
collectedBytes integer yes Bytes collected since last report.
collectedEvents integer yes Events collected since last report.
collectedBatches integer yes Batches collected since last report.
parser string false Parser used.
repository string false The repository ingested into.
errors array yes Array of errors.

Each entry in errors:

Field Type Required Description
sourceIdentifier string yes Identifier of the source that errored.
error string yes Error message.

Kind problemsReport

The following is an example of a problemsReport event: 

{
  "kind":"problemsReport",
  "id":"XWt9NuWvgPpjhMQmtXAoQ6NCoAFRddpr",
  "timestamp":1777556112043,
  "problems":[
  0: {
  "category":"pipeline",
  "error":"Post \"url\": dial tcp: lookup url: no such host","message":"Could not send data to sink \"logscale\". Sending will be retried."}]
}
Field Type Required Description
kind constant problemsReport yes The kind field determines the type of event.
id string yes The globally unique ID assigned to the enrolled Falcon LogScale Collector. Used to correlate the metrics events.
timestamp integer yes The timestamp of the event in epoch milliseconds.
problems array yes Array of problem entries.

Each entry in problems:

Field Type Required Description
message string yes Description of the problem.
error string no Error details (omitted if not present).
category string yes Classification — one of "logsources", "pipeline", "fleetmanagement", "update-service".

Kind ingest

The following is an example of an ingest event: 

{
  "kind":"ingest",
  "id":"XWt9NuWvgPpjhMQmtXAoQ6NCoAFRddpr",
  "bucketStart":1777636498401,
  "bucketEnd":1777636558400,
  "sum":48144
}
Field Type Required Description
kind constant ingest yes The kind field determines the type of event.
id string yes The globally unique ID assigned to the enrolled Falcon LogScale Collector. Used to correlate the metrics events.
timestamp integer yes The timestamp of the event in epoch milliseconds.
bucketStart integer yes Start of the time bucket (Unix ms).
bucketEnd integer yes End of the time bucket (Unix ms).
sum integer yes Total ingested bytes during the bucket period.