Pagination of Results
Search results can be obtained 'around' a specific event ID from a
result set using the
around
parameter. This is a sub-structure to the main QueryInput JSON structure
when making a request.
Table: Around JSON Object Fields
Field | Type | Required? | Default | Description |
---|---|---|---|---|
eventId | string | Yes | The ID of the event to use as the reference point | |
numberOfEventsAfter | integer | Yes | Number of events to show after the eventId | |
numberOfEventsBefore | integer | Yes | Number of events to show before the eventId | |
timestamp | integer | Yes | The timestamp to use as the reference for pagination. |
Querying using this method is a two-stage process; first find a reference ID of the query around which you want to view matching events, then search again specifying the number of events before and after that reference event.
For example:
{
"queryString" : "css",
"start" : "1year"
}
curl -v -X POST https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs \
-H "Accept: application/x-ndjson" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- << EOF
{
"start" : "1year",
"queryString" : "css"
}
EOF
curl -v -X POST https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs \
-H "Accept: application/x-ndjson" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- << EOF
{
\"queryString\" : \"css\",
\"start\" : \"1year\"
}
EOF
curl -v -X POST https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs ^
-H "Accept: application/x-ndjson" ^
-H "Authorization: Bearer $TOKEN" ^
-H "Content-Type: application/json" ^
-d @'{ ^
\"queryString\" : \"css\", ^
\"start\" : \"1year\" ^
} ^
'
curl.exe -X POST
-H "Accept: application/x-ndjson"
-H "Authorization: Bearer $TOKEN"
-H "Content-Type: application/json"
-d '{
\"queryString\" : \"css\",
\"start\" : \"1year\"
}
'
"https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs"
#!/usr/bin/perl
use HTTP::Request;
use LWP;
my $INGEST_TOKEN = "TOKEN";
my $uri = 'https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs';
my $json = '{
\"start\" : \"1year\",
\"queryString\" : \"css\"
}
';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Accept" => "application/x-ndjson");
$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";
#! /usr/local/bin/python3
import requests
url = 'https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs'
mydata = r'''{
"queryString" : "css",
"start" : "1year"
}
'''
resp = requests.post(url,
data = mydata,
headers = {
"Accept" : "application/x-ndjson",
"Authorization" : "Bearer $TOKEN",
"Content-Type" : "application/json"
}
)
print(resp.text)
const https = require('https');
const data = JSON.stringify(
{
\"queryString\" : \"css\",
\"start\" : \"1year\"
}
);
const options = {
hostname: 'https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs',
path: '/graphql',
port: 443,
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': data.length,
Authorization: 'BEARER ' + process.env.TOKEN,
'User-Agent': 'Node',
},
};
const req = https.request(options, (res) => {
let data = '';
console.log(`statusCode: ${res.statusCode}`);
res.on('data', (d) => {
data += d;
});
res.on('end', () => {
console.log(JSON.parse(data).data);
});
});
req.on('error', (error) => {
console.error(error);
});
req.write(data);
req.end();
The query will return matching results with each event containing a unique ID:
@timestamp | #humioBackfill | #repo | #type | @host | @id | @ingesttimestamp | @source | @timestamp.nanos | @timezone |
---|---|---|---|---|---|---|---|---|---|
2023-03-07T15:09:42 | 0 | weblog | kv | ML-C02FL14GMD6V | XPcjXSqXywOthZV25sOB1hqZ_0_1_1678201782 | 2023-08-08T08:31:23 | /var/log/apache2/access_log | 0 | Z |
2023-03-07T15:09:43 | 0 | weblog | kv | ML-C02FL14GMD6V | XPcjXSqXywOthZV25sOB1hqZ_0_3_1678201783 | 2023-08-08T08:31:23 | /var/log/apache2/access_log | 0 | Z |
2023-03-09T14:16:56 | 0 | weblog | kv | ML-C02FL14GMD6V | XPcjXSqXywOthZV25sOB1hqZ_0_15_1678371416 | 2023-08-08T08:31:23 | /var/log/apache2/access_log | 0 | Z |
2023-03-09T14:16:59 | 0 | weblog | kv | ML-C02FL14GMD6V | XPcjXSqXywOthZV25sOB1hqZ_0_22_1678371419 | 2023-08-08T08:31:23 | /var/log/apache2/access_log | 0 | Z |
2023-03-09T14:16:59 | 0 | weblog | kv | ML-C02FL14GMD6V | XPcjXSqXywOthZV25sOB1hqZ_0_23_1678371419 | 2023-08-08T08:31:23 | /var/log/apache2/access_log | 0 | Z |
Then, identify the ID (in the @id
field of the response) of the original event to use as the original
query, and the timespan that must be provided along with the
around
object that defines the scope, in this case 100 events before and after
the reference event.
{
"around" : {
"eventId" : "XPcjXSqXywOthZV25sOB1hqZ_0_23_1678371419",
"numberOfEventsBefore" : 100,
"timestamp" : 1678371419000,
"numberOfEventsAfter" : 100
},
"start" : "1year",
"queryString" : "css"
}
curl -v -X POST https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs \
-H "Accept: application/x-ndjson" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- << EOF
{
"queryString" : "css",
"around" : {
"numberOfEventsAfter" : 100,
"eventId" : "XPcjXSqXywOthZV25sOB1hqZ_0_23_1678371419",
"timestamp" : 1678371419000,
"numberOfEventsBefore" : 100
},
"start" : "1year"
}
EOF
curl -v -X POST https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs \
-H "Accept: application/x-ndjson" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- << EOF
{
\"around\" : {
\"numberOfEventsAfter\" : 100,
\"eventId\" : \"XPcjXSqXywOthZV25sOB1hqZ_0_23_1678371419\",
\"numberOfEventsBefore\" : 100,
\"timestamp\" : 1678371419000
},
\"start\" : \"1year\",
\"queryString\" : \"css\"
}
EOF
curl -v -X POST https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs ^
-H "Accept: application/x-ndjson" ^
-H "Authorization: Bearer $TOKEN" ^
-H "Content-Type: application/json" ^
-d @'{ ^
\"around\" : { ^
\"numberOfEventsAfter\" : 100, ^
\"eventId\" : \"XPcjXSqXywOthZV25sOB1hqZ_0_23_1678371419\", ^
\"timestamp\" : 1678371419000, ^
\"numberOfEventsBefore\" : 100 ^
}, ^
\"start\" : \"1year\", ^
\"queryString\" : \"css\" ^
} ^
'
curl.exe -X POST
-H "Accept: application/x-ndjson"
-H "Authorization: Bearer $TOKEN"
-H "Content-Type: application/json"
-d '{
\"queryString\" : \"css\",
\"around\" : {
\"numberOfEventsBefore\" : 100,
\"timestamp\" : 1678371419000,
\"eventId\" : \"XPcjXSqXywOthZV25sOB1hqZ_0_23_1678371419\",
\"numberOfEventsAfter\" : 100
},
\"start\" : \"1year\"
}
'
"https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs"
#!/usr/bin/perl
use HTTP::Request;
use LWP;
my $INGEST_TOKEN = "TOKEN";
my $uri = 'https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs';
my $json = '{
\"queryString\" : \"css\",
\"start\" : \"1year\",
\"around\" : {
\"eventId\" : \"XPcjXSqXywOthZV25sOB1hqZ_0_23_1678371419\",
\"numberOfEventsBefore\" : 100,
\"timestamp\" : 1678371419000,
\"numberOfEventsAfter\" : 100
}
}
';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Accept" => "application/x-ndjson");
$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";
#! /usr/local/bin/python3
import requests
url = 'https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs'
mydata = r'''{
"queryString" : "css",
"around" : {
"numberOfEventsAfter" : 100,
"numberOfEventsBefore" : 100,
"timestamp" : 1678371419000,
"eventId" : "XPcjXSqXywOthZV25sOB1hqZ_0_23_1678371419"
},
"start" : "1year"
}
'''
resp = requests.post(url,
data = mydata,
headers = {
"Accept" : "application/x-ndjson",
"Authorization" : "Bearer $TOKEN",
"Content-Type" : "application/json"
}
)
print(resp.text)
const https = require('https');
const data = JSON.stringify(
{
\"queryString\" : \"css\",
\"start\" : \"1year\",
\"around\" : {
\"timestamp\" : 1678371419000,
\"numberOfEventsBefore\" : 100,
\"eventId\" : \"XPcjXSqXywOthZV25sOB1hqZ_0_23_1678371419\",
\"numberOfEventsAfter\" : 100
}
}
);
const options = {
hostname: 'https://$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/queryjobs',
path: '/graphql',
port: 443,
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': data.length,
Authorization: 'BEARER ' + process.env.TOKEN,
'User-Agent': 'Node',
},
};
const req = https.request(options, (res) => {
let data = '';
console.log(`statusCode: ${res.statusCode}`);
res.on('data', (d) => {
data += d;
});
res.on('end', () => {
console.log(JSON.parse(data).data);
});
});
req.on('error', (error) => {
console.error(error);
});
req.write(data);
req.end();
This will return the events around the reference event:
@timestamp | #repo | #type | @error | @error_msg | @error_msg[0] | @error_msg[1] | @host | @id | @ingesttimestamp | @source | @sourcetype |
---|---|---|---|---|---|---|---|---|---|---|---|
2024-01-11T11:54:21 | weblog | customjson | true | Could not parse json for field=@rawstring msg=Could not handle input. reason=Could not parse JSON | No timestamp found in field "@rawstring". | Could not parse json for field=@rawstring msg=Could not handle input. reason=Could not parse JSON | No timestamp found in field "@rawstring". | Ea5aqq0vOp3l2NPzpqCWfQSK_0_0_1704974061 | 2024-01-11T11:54:21 | apache/http-server:apache_access | ||
2023-10-19T09:51:33 | weblog | accesslog | true | The event was filtered out by the parser. The original input is available in the `@rawstring` field of this event. | The event was filtered out by the parser. The original input is available in the `@rawstring` field of this event. | ML-C02FL14GMD6V | 0kY6kpTPP9kbGiYSUWP5mMhg_0_22_1697709093 | 2023-10-19T09:51:33 | /var/log/apache2/public-access_log | ||
2023-03-07T15:09:43 | weblog | kv | ML-C02FL14GMD6V | XPcjXSqXywOthZV25sOB1hqZ_0_3_1678201783 | 2023-08-08T08:31:23 | /var/log/apache2/access_log | |||||
2023-03-07T15:09:42 | weblog | kv | ML-C02FL14GMD6V | XPcjXSqXywOthZV25sOB1hqZ_0_1_1678201782 | 2023-08-08T08:31:23 | /var/log/apache2/access_log |
The
around
query functionality is also used via the UI (see
Searching Data).