The humio Repository

The humio repository is a system repository that holds debug logs generated by LogScale internally. It also has a few built-in dashboards to help you quickly ascertain the state of your LogScale cluster or server.

LogScale logging types are listed below with the names of their respective log files, whether they are for cloud or self-hosted environments, and in which repositories they are available. All of the logs are available in the humio repository , so they have #type=humio. They will also have a #kind tag for each that corresponds to the log type column in the table below. The humio Repository contains the following #kind events:

Log type Log file Cloud Self-hosted Description Repository available
logs humio-debug.log No Yes humio debug logs. Not available on Cloud. See humio
activity humio-activity.log No Yes Logs of relevant user activity. For self-hosted. See humio-activity or humio
activity humio-activity.log Yes No Logs of relevant user activity. For Cloud. humio-organization-activity
usage humio-usage.log No Yes Logs of aggregated measurements data. For self-hosted. humio-usage
metrics humio-metrics.log No Yes Contains a broad variety of metrics from internal LogScale runtime for Self-Hosted. See humio-metrics or humio
metrics humio-metrics.log Yes No Contains a broad variety of metrics from internal LogScale runtime for Cloud. humio-organization-metrics
requests humio-requests.log No Yes All HTTP requests. Like an accesslog in LogScale's own format. See humio
nonsensitive humio-non-sensitive.log No Yes Selected log lines where no searches or user data will be present. This can be shipped to LogScale support or other parties. See humio
threaddumps humio-threaddumps.log No Yes LogScale's regular log threaddumps. See humio

The above logs are automatically rotated by LogScale when they reach 50 megabytes in size. LogScale retains up to five files of each.

All of the above logs are available to search in the humio repository. When searching LogScale logs in the humio repository, the tag #type, #kind, and #vhost can be used.

Note

From version 1.255, the usage logs in the humio repository use #kind=usage instead of #kind=logs. This will break existing queries that query these logs using #kind=logs.

For those logs which are available in other repositories, you can see information about the logs and their fields in their respective chapters.

Important

Data in system repositories cannot be redacted. For more information about immutability of data, see Immutability of data.

For more information about logging LogScale to LogScale, see LogScale Internal Logging

Each #kind has fields associated with it.

Humio Repository log examples

The following contains examples of logs from the humio repository.

Activity event example

An example of an activity event is for a trigger. So the information contained in the event is relevant for trigger execution.

Field Value
#kind Activity
#repo humio
#type humio
#vhost 8
@host example.testing:8080
@id XQP9NSlmxlxz6nHCuvRFgiDQ_113_111_1684918557
@ingesttimestamp 1731981792574
@timestamp 1731981792574
@timestamp.nanos 0
@timezone Z
category ScheduledSearch
dataspace humio
externalQueryId P1-q4u0GQwR6Xel6XKT1HAMv8Ts
message Scheduled search found no results and will not trigger
plannedExecutionTime 1731981780000
queryFinishedTime 1731981792573
queryIntervalEndTime 1731981780000
queryIntervalStartTime 1731376980000
scheduledSearchId ipIXwhHy1a1MrBqujswy3OITZkLUpO0k
scheduledSearchName FLTR aidmaster Generation
severity Info
source console.log
status Success
subCategory Query
timestamp 1731981792574
viewId KFrfTEli7ziKVdJiHMzmy6AV

The @rawstring contains the raw event log data that has been parsed into the fields.

Usage event example

A usage event records aggregated measurements data at the organization level about ingest volume, storage size, and contracted limits. These events can be useful when monitoring resource consumption or comparing usage against your contracted thresholds.

Field Example Value Explanation
#repo humio-usage Repository the usage related logs come from.
#sampleRate hour To which period the values in this log pertain to. 1 hour in most cases.
#sampleType organization The aggregation level for the measurements in this log line. In the humio repository, this is always organization.
@id ...Kz_467_7_1759133983... ID of the event
averageDailyIngestAfterFieldRemovalSizeOver30Days 11630390754468 Average daily data volume ingested after field removal processing over the past 30 days
averageDailyDataScannedOver30Days 233157158845848 Average amount of data scanned daily when running queries over the past 30 days
averageDailySegmentWriteBytesOver30Days 6413599076174 Average daily volume of data written to segments over the past 30 days for the log line. May differ from raw ingestion volumes due to compression, field removal, and other storage optimizations.
cid   32-character (hex) customer identifier number
collectorFieldsSize 54182037 Size of fields data collected by LogScale collectors before being sent to LogScale. Represents the volume of metadata and field information gathered at the collection point. Can help users understand how much of their data volume is attributed to fields/metadata versus raw log content. Excessive field size might indicate opportunities to optimize collection configurations by removing unnecessary fields at the source, which could improve overall system efficiency and reduce costs. Related to the size of the Log Collector ingested metadata fields which all are not part of the averageDailyDataScannedOver30Days.
contractedDailyIngestBase10 9999999000000000 Represents the organization's contracted daily ingest limit, converted to bytes using base 10. This is the value that should be used when comparing the customer's actual daily ingest to their contracted limit.
contractedRetention 9223372036854775807 Agreed-upon data retention period specified in a LogScale contract or service agreement. Retention might be different on certain repositories.
contractedUsers 999999 Maximum number of user licenses included in the LogScale contract or service agreement
dataScanned 123546 The amount of data that was scanned in the last hour by the customer in #sampleType.
falconCollectorFieldsSize 54182037 Size of fields data specifically collected by Falcon sensors/collectors before being sent to LogScale. Distinct from general collectorFieldsSize as it focuses only on Falcon-sourced data. This metric can help identify opportunities to optimize Falcon data collection by adjusting field collection settings if the metadata volume is unexpectedly high. Related to the size of the Log Collector ingested metadata fields which are not part of the falconAverageDailyDataScannedOver30Days.
falconIngestAfterFieldRemovalSize 12311214 Volume of data ingested specifically from Falcon sensors after field removal processing has been applied. Distinct from general ingest metrics as it focuses only on CrowdStrike Falcon-sourced data. By comparing this to raw Falcon ingest volumes, teams can assess the effectiveness of their field removal strategies specifically for security telemetry data.
falconIngestBytes 23123 Total raw volume of data (unprocessed data size before any field removal or compression) ingested from CrowdStrike Falcon sensors/endpoints. Measures specifically Falcon-sourced data. Security teams can use this to track trends in endpoint telemetry volume, identify unusual spikes in security data, and ensure they have adequate capacity for their security monitoring needs. When compared with falconIngestAfterFieldRemovalSize, it helps quantify the effectiveness of data optimization strategies specifically for security telemetry.
falconProcessedEventsSize 612352905 Amount of Falcon event data after initial processing but before storage optimization. Represents the size of Falcon telemetry after parsing, normalization, and enrichment, but before compression and field removal. This metric helps security teams understand how their Falcon data changes during the processing pipeline. By comparing falconIngestBytes (raw data coming in), falconProcessedEventsSize (data after processing), falconIngestAfterFieldRemovalSize (data after field removal), and falconStorageSize (final stored data), teams can identify where in the pipeline data volume changes significantly and optimize accordingly. For example, if processed events are much larger than raw ingest, it might indicate excessive enrichment or parsing that could be streamlined. This metric is particularly valuable for tuning the performance and efficiency of security data processing.
falconRemovedFieldsSize 48493 Total volume of field data that has been removed from CrowdStrike Falcon telemetry during ingestion. Shows how much data you've prevented from being stored by dropping unnecessary fields.
falconSegmentWriteBytes 12313214 Amount of data in bytes written to storage segments specifically for CrowdStrike Falcon telemetry.
falconStorageSize 129071068836 Total storage space used to store data from CrowdStrike Falcon sensors/endpoints.
ignoreContract false Boolean flag indicating whether the contract was ignored
ingestAfterFieldRemovalSize 12311214 Measures the size of log data after specific fields have been removed during the ingestion process. This is the LogScale data ingest volume as it counts towards license usage. The value is calculated as ingestAfterFieldRemovalSize = processedEvent - removedFields. For detailed information, see Measure Data Ingest.
ingestBytes 23123 The amount of data that was ingested to this #sampleType in the last #sampleRate, measured in bytes.
logId 17591123 The ID that binds the logs with different #sampleType together. See LogId in humio-usage repository .
measurementPoint ProcessedEventsSize The measurement type used for the event in the log line.
measurementsQueryUrl https://your-logscale-instance.com/humio-measurements/search?query=orgId%20%3D%20SINGLE_ORGANIZATION_ID%0A%7C%20dataspaceId%20%3D%20YOUR_DATASPACE_ID%0A%7C%20groupBy%28%22%23measurement%22%2C%20function%3Dsum%28byteCount%29%29&live=false&start=1234567890123&end=1234567890456 The query used in the humio-measurements repository to generate this log line.
missingContract false Boolean flag indicating whether a valid contract is associated with LogScale. False indicates the contract exists. True indicates the contract is missing.
orgId   The organization ID that the measurements in this log line pertain to.
orgName MyOrgName The organization name that the measurements in this log line pertain to, if #sampleType is organization.
processedEventsSize 612352905 Amount of event data after initial processing but before storage optimization. Represents the size after parsing, normalization, and enrichment, but before compression and field removal. This metric helps security teams understand how their data changes during the processing pipeline. By comparing ingestBytes (raw data coming in), processedEventsSize (data after processing), ingestAfterFieldRemovalSize (data after field removal), and storageSize (final stored data), teams can identify where in the pipeline data volume changes significantly and optimize accordingly. For example, if processed events are much larger than raw ingest, it might indicate excessive enrichment or parsing that could be streamlined. This metric is particularly valuable for tuning the performance and efficiency of data processing.
queryEnd 2021-06-28T07:31:23.044Z The time window end of querying the humio-measurements repository .
queryStart 2021-06-28T07:31:23.044Z The time window beginning of querying the humio-measurements repository .
removedFieldSize 48493 Size of fields removed in the log line.
segmentWriteBytes 12313214 The amount of data in bytes written to the disk in the last hour.
storageSize 129071068836 Total disk usage in the #sampleType.
subscription Paying Indicates which LogScale subscription type the log line data belongs to
userCount 48 Total number of user accounts currently configured

Logs event example

Logs events contain LogScale's own logs so the event information is relevant to these types of logs, such as thread, loglevel, and exception.

Field Value
#kind logs
#repo humio
#type humio
#vhost 9
@host example.testing:8080
@id T3PaSn7CE245lnoBvLnKD53I_1574_489_1732019470
@ingesttimestamp 1732019470846
@timestamp 1732019470845
@timestamp.nanos 0
@timezone Z
class c.h.s.e.QueryEndpoint
dataspace humio
exception org.apache.pekko.stream.StreamTcpException:
hostId 2
loglevel WARN
query #kind
source console.log
thread humio-pekko.actor.default-dispatcher-8
viewId nrWby53THW3pSm7RAzWXTs4y

The @rawstring contains the raw event log data that has been parsed into the fields.

Threaddumps event example

A threaddump event captures information about system performance. These events can be useful when troubleshooting system performance or optimization issues.

Field Value
#kind threaddumps
#repo humio
#type humio
#vhost 9
@host example.testing:8080
@id xodbp9Lf4BC8Xcdz07LIMhRv_645_127_1732018811
@ingesttimestamp 1732018811599
@timestamp 1732018811592
@timestamp.nanos 0
@timezone Z
dumpedAt 1732018811578
dumpInterval 10000
group segment-merger
humioClass com.humio.util.ExecutorsUtil$HumioThreadFactory
humioFunction idleWaitInThreadPool$1
humioLine app//com.humio.util.ExecutorsUtil$HumioThreadFactory.idleWaitInThreadPool$1(ExecutorsUtil.scala:95)
humioLineNo 95
name segment-merger-0
source console.log
state WAITING
tid 307

The @rawstring contains the raw event log data that has been parsed into the fields.

Non-sensitive event example

Non-sensitive events show selected log lines where no searches or user data will be present.

Field Value
#kind nonsensitive
#repo humio
#type humio
#vhost 9
@host example.testing:8080
@id 3U0GkIoAVPacOWHI6vhjwLkN_44_33_1732019115
@ingesttimestamp 1732019115518
@timestamp 1732019115518
@timestamp.nanos 0
@timezone Z
device eth0
name netdev
rx_bytes 5715748266
rx_compressed 0
rx_drop 0
rx_errs 0
rx_fifo 0
rx_frame 0
rx_multicast 0
rx_packets 4767385
source console.log
tx_bytes 4337916381
tx_carrier 0
tx_colls 0
tx_compressed 0
tx_drop 0
tx_errs 0
tx_fifo 0
tx_packets 3608874

The @rawstring contains the raw event log data that has been parsed into the fields.

Requests event example

A requests events show all HTTP requests, similar to an accesslog in LogScale's own format.

Field Value
#kind requests
#repo humio
#type humio
#vhost 9
@host example.testing:8080
@id a5MyAtLBaWXToRAQG0hSpVhV_14_250_1732008410
@ingesttimestamp 1732008410584
@timestamp 1732008410584
@timestamp.nanos 0
@timezone Z
connectionSource 10.244.6.1:55464
contentLength 0
decodedContentLength 0
internal false
method GET
remote 10.244.6.1:55464
responseLength 0
route humio
source console.log
status 200
time 0
uri http://10.244.6.72:8080/api/v1/is-node-up
userAgent kube-probe/1.20

The @rawstring contains the raw event log data that has been parsed into the fields.

Metrics event example

A metrics event is also shown in the humio-metrics Repository for self-hosted or humio-organization-metrics Repository for Cloud.

Field Value
#kind metrics
#repo humio
#type humio
#vhost 9
@host example.testing:8080
@id AfBZBFb7QXJhxe78wWvcV2x9_104_1195_1732010069
@ingesttimestamp 1732010069740
@timestamp 1732010069686
@timestamp.nanos 0
@timezone Z
count 0
duration_unit milliseconds
m1 0.0
m15 0.0
m5 0.0
max 0.0
mean 0.0
mean_rate 0.0
median 0.0
metric_type TIMER
min 0.0
name serialize-state-time
p75 0.0
p95 0.0
p98 0.0
p99 0.0
p999 0.0
rate_unit events/second
source console.log
stddev 0.0

The @rawstring contains the raw event log data that has been parsed into the fields. Note that some of the fields shown here are described in The humio-metrics Repository.