Manage Digest Partitions

Digest partitions route the incoming data while it is "in progress".

For more information, read the Digest Rules documentation.

Warning

Do not POST to this API unless the cluster is running fine, with all members connected and active. All digests stop for a few seconds when being applied.

Digest does not start before all nodes are ready, thus if a node is failing, digest does not resume.

Show digest partition information

GET/POST the setting to hand-edit where each partition goes. You cannot reduce the number of partitions.

Description	Returns a list of the digest partitions.
Method	`GET /api/v1/clusterconfig/ingestpartitions`
Authentication Required	yes
Return Codes
200	Request complete
400	Bad authentication
500	Request failed

http

GET /api/v1/clusterconfig/ingestpartitions

To redirect the output to a JSON file, run the script like this:

shell

$ script>$YOUR_FILE_NAME.json

You can then use this as the payload in further requests when updating the information.

Mac OS or Linux (curl)

shell

curl -v -X GET $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json"

Mac OS or Linux (curl) One-line

shell

curl -v -X GET $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json"

Windows Cmd and curl

cmd

curl -v -X GET $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions ^
    -H "Authorization: Bearer $TOKEN" ^
    -H "Content-Type: application/json"

Windows Powershell and curl

powershell

curl.exe -X GET 
    -H "Authorization: Bearer $TOKEN"
    -H "Content-Type: application/json"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions';
my $req = HTTP::Request->new("GET", $uri );
$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions'

resp = requests.get(url,
                     headers = {
   "Authorization" : "Bearer $TOKEN",
   "Content-Type" : "application/json"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');

let request = https.get('$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions', (res) => {
  if (res.statusCode !== 200) {
    console.error(`Error from server. Code: ${res.statusCode}`);
    res.resume();
    return;
  }

  let data = '';

  res.on('data', (chunk) => {
    data += chunk;
  });

  res.on('close', () => {
    console.log('Response:');
    console.log(JSON.parse(data));
  });
});

    return(undef,undef);
}

Update digest partition settings

Description	Manage the digest partitions to control manually where each partition is.
Method	`POST /api/v1/clusterconfig/ingestpartitions`
Request Data
Authentication Required	yes
Return Codes
200	Request complete
400	Bad authentication
500	Request failed

http

POST /api/v1/clusterconfig/ingestpartitions

Mac OS or Linux (curl)

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '@$YOUR_FILE_NAME.json'

Mac OS or Linux (curl) One-line

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '@$YOUR_FILE_NAME.json'

Windows Cmd and curl

cmd

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions ^
    -H "Authorization: Bearer $TOKEN" ^
    -H "Content-Type: application/json" ^
    -d '@$YOUR_FILE_NAME.json'

Windows Powershell and curl

powershell

curl.exe -X POST 
    -H "Authorization: Bearer $TOKEN"
    -H "Content-Type: application/json"
    -d '@$YOUR_FILE_NAME.json'
"$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions';
my $json = '@$YOUR_FILE_NAME.json';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions'
mydata = r'''@$YOUR_FILE_NAME.json'''

resp = requests.post(url,
                     data = mydata,
                     headers = {
   "Authorization" : "Bearer $TOKEN",
   "Content-Type" : "application/json"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');

const data = JSON.stringify(
    @$YOUR_FILE_NAME.json
);


const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();

Update digest partition settings using defaults

Invoke setdefaults to distribute the current number of partitions evenly among the known nodes in the cluster

Description	Control partition distribution to nodes in the cluster.
Method	`POST /api/v1/clusterconfig/ingestpartitions/setdefaults`
Request Data
Authentication Required	yes
Return Codes
200	Request complete
400	Bad authentication
500	Request failed

http

POST /api/v1/clusterconfig/ingestpartitions/setdefaults

Mac OS or Linux (curl)

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '@$YOUR_FILE_NAME.json'

Mac OS or Linux (curl) One-line

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '@$YOUR_FILE_NAME.json'

Windows Cmd and curl

cmd

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults ^
    -H "Authorization: Bearer $TOKEN" ^
    -H "Content-Type: application/json" ^
    -d '@$YOUR_FILE_NAME.json'

Windows Powershell and curl

powershell

curl.exe -X POST 
    -H "Authorization: Bearer $TOKEN"
    -H "Content-Type: application/json"
    -d '@$YOUR_FILE_NAME.json'
"$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults';
my $json = '@$YOUR_FILE_NAME.json';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults'
mydata = r'''@$YOUR_FILE_NAME.json'''

resp = requests.post(url,
                     data = mydata,
                     headers = {
   "Authorization" : "Bearer $TOKEN",
   "Content-Type" : "application/json"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');

const data = JSON.stringify(
    @$YOUR_FILE_NAME.json
);


const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();