Show Offline Nodes

Show the list of available nodes currently in an offline state

Query

flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2["Expression"] 3{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> result
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2["Expression"] 3{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> result
logscale
#type=humio #kind=logs class=/ClusterHostAliveStats/  "AliveStats on me"
age > 7200000 /* =2hours */
timeChart(hostId, function=count(hostId,distinct=true), limit=50, minSpan=4h)

Introduction

The timeChart() function can be used to aggregate events into time-based buckets and display the results as a time series, grouped by a specified field. By combining timeChart() with count() using the distinct parameter, it is possible to count unique occurrences of a field value within each time bucket rather than counting all events. This makes timeChart() well suited for monitoring queries where the goal is to track how many distinct entities, such as nodes or hosts, are in a particular state across consecutive time intervals.

"Node Offline" events within LogScale are generated when a node is reported offline by the other nodes in the cluster. This query shows Offline Nodes.

Example incoming data might look like this:

@timestamp#type#kindclasshostIdage@rawstring
1742032800000humiologscom.humio.ClusterHostAliveStatshost-019200000AliveStats on me host-01 reporting cluster status
1742032800000humiologscom.humio.ClusterHostAliveStatshost-023100000AliveStats on me host-02 reporting cluster status
1742032800000humiologscom.humio.ClusterHostAliveStatshost-038500000AliveStats on me host-03 reporting cluster status
1742032800000humiologscom.humio.ClusterHostAliveStatshost-041200000AliveStats on me host-04 reporting cluster status
1742032800000humiologscom.humio.ClusterHostAliveStatshost-057900000AliveStats on me host-05 reporting cluster status
1742032800000humiologscom.humio.ClusterHostAliveStatshost-06500000AliveStats on me host-06 reporting cluster status
1742032800000humiologscom.humio.IngestPipelinehost-019200000Ingest pipeline processing on host-01
1742032800000humiometricscom.humio.ClusterHostAliveStatshost-029200000AliveStats on me host-02 metrics report
1742047200000humiologscom.humio.ClusterHostAliveStatshost-019800000AliveStats on me host-01 reporting cluster status
1742047200000humiologscom.humio.ClusterHostAliveStatshost-022800000AliveStats on me host-02 reporting cluster status
1742047200000humiologscom.humio.ClusterHostAliveStatshost-037900000AliveStats on me host-03 reporting cluster status
1742047200000humiologscom.humio.ClusterHostAliveStatshost-04900000AliveStats on me host-04 reporting cluster status
1742047200000humiologscom.humio.ClusterHostAliveStatshost-058100000AliveStats on me host-05 reporting cluster status
1742047200000humiologscom.humio.ClusterHostAliveStatshost-06400000AliveStats on me host-06 reporting cluster status
1742047200000humiologscom.humio.QueryServicehost-039100000Query service heartbeat on host-03
1742047200000humiometricscom.humio.ClusterHostAliveStatshost-058100000AliveStats on me host-05 metrics report
1742061600000humiologscom.humio.ClusterHostAliveStatshost-0110200000AliveStats on me host-01 reporting cluster status
1742061600000humiologscom.humio.ClusterHostAliveStatshost-023400000AliveStats on me host-02 reporting cluster status
1742061600000humiologscom.humio.ClusterHostAliveStatshost-038800000AliveStats on me host-03 reporting cluster status
1742061600000humiologscom.humio.ClusterHostAliveStatshost-041100000AliveStats on me host-04 reporting cluster status
1742061600000humiologscom.humio.ClusterHostAliveStatshost-057900000AliveStats on me host-05 reporting cluster status
1742061600000humiologscom.humio.ClusterHostAliveStatshost-06600000AliveStats on me host-06 reporting cluster status
1742061600000humiologscom.humio.IngestPipelinehost-048800000Ingest pipeline processing on host-04

Step-by-Step

  1. Starting with the source repository events.

  2. flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2["Expression"] 3{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> result style 1 fill:#ff0000,stroke-width:4px,stroke:#000;
    flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2["Expression"] 3{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> result style 1 fill:#ff0000,stroke-width:4px,stroke:#000;
    logscale
    #type=humio #kind=logs class=/ClusterHostAliveStats/  "AliveStats on me"

    Filters on all logs in humio repository that are tagged with kind equal to logs and then returns the events where the class field has values containing /ClusterHostAliveStats/, and where the logline contains the string AliveStats on me.

  3. flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2["Expression"] 3{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> result style 2 fill:#ff0000,stroke-width:4px,stroke:#000;
    flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2["Expression"] 3{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> result style 2 fill:#ff0000,stroke-width:4px,stroke:#000;
    logscale
    age > 7200000 /* =2hours */

    Returns all events where the values of the field age is greater than 7200000 ms. Notice that this example uses multi-line comments /* =2hours */ to help describe the value which we can describe in more detail by looking at each stage of the calculation as shown below:

    none
    7200000ms / 1000 # 7200 seconds
/ 60 # 120 minutes
/ 60 # 2 hours
= 2
  4. flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2["Expression"] 3{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> result style 3 fill:#ff0000,stroke-width:4px,stroke:#000;
    flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2["Expression"] 3{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> result style 3 fill:#ff0000,stroke-width:4px,stroke:#000;
    logscale
    timeChart(hostId, function=count(hostId,distinct=true), limit=50, minSpan=4h)

    Counts the events grouping by the field hostId, creating an aggregate list and displaying the last 50 returned results in buckets of 4 hours in a Time Chart.

  5. Event Result set.

Summary and Results

The query is used to show a list of available nodes currently in an offline state.

This query is useful, for example, to track which nodes in a LogScale cluster have been consistently offline over time, enabling operations teams to identify persistent infrastructure problems.

Sample output from the incoming example data:

_buckethost-01host-03host-05
1742032800000111
1742047200000111
1742061600000111

Note that host-02, host-04, and host-06 are excluded from the results in all buckets because their age values never exceed 7200000 milliseconds.