LogScale Measurement Repositories

We log your data volume in multiple repositories. You can use them to run audits to see how much data you ingest, which repositories it went to, and how much are you storing.

humio-organization-usage View

The humio-organization-usage view is available to Cloud customers, and contains data from two repositories, humio-measurements and humio-usage.

The humio-organization-usage view contains logs with information on how much data you are ingesting to LogScale, how much data you have stored, and in which repositories. It also tells you how much data you are scanning when searching through logs.

You can filter the logs by which repository they come from by using the repositoryName field in the humio-measurements repo or the repo field in the humio-usage repository.

If you have LogScale self-hosted license, you have access to these repositories directly, and because of that, do not have humio-organization-usage view.

The following query in your humio-organization-usage View is visualized on the Usage Page as your license consumption:

logscale
#measurement = "processed_events"
| repositoryName="*"
| sum(field = byteCount)

To see usage for a specific repository, you can replace the "*" with a specific repository title.

If your order form states that we define data volume as "raw (uncompressed) data ingested by the product", we derive your usage of the license volume from the segmentWriteBytes metric instead. This is the amount of data in bytes written to the disk.

humio-usage Repository

The logs in this repository are the results of an hourly query to the humio-measurements repository. It differs from the humio-measurements repository in the following: it has unlimited retention, data is being logged once every hour, and it does not include data on ingestion source. Moreover, the usage measurements are provided as fields in the log.

In the table below, there are some of the more interesting fields a log line could have:

Field Example Value Explanation
#processedEventsSize 124534 The size of an event ingested by the LogScale platform, in bytes
#sampleRate hour To which period the values in this log pertain to. 1 hour in most cases.
#sampleType usageTag If this log line refers to a repository, or a set of repositories that are grouped under the same usageTag. The value can be one of the following: organization, usageTag or repository.
repo your_repo_name The repository name measurements in this log line pertain to, if #sampleType is repository.
dataScanned 123546 The amount of data that was scanned in the last hour in #sampleType.
ingestBytes 23123 The amount of data that was ingested to this #sampleType in the last #sampleRate, measured in bytes.
segmentWriteBytes 12313214 The amount of data in bytes written to the disk in the last hour.
storageSize 129071068836 Total disk usage in the #sampleType.
queryStart 2021-06-28T07:31:23.044Z The time window beginning of querying the humio-measurements repository.
queryEnd 2021-06-28T07:31:23.044Z The time window end of querying the humio-measurements repository.
logId 21 The id that binds the logs of different #sampleType together. See the section on LogId below.
LogId in LogScale Usage Repository

The logs with different #sampleTypes share one value, which is the logId.

#sampleType processedEventsSize logId
repository 2909 2
repository 1290 2
repository 879 2
organization 5078 2

By tracing the logId, you can drill down into your usage, and find out what your usage was in a specific time period, down to an hour, by repository. Since there is unlimited retention on this repository, you will always be able to see your usage from the beginning of your usage of LogScale.