Manage Digest Partitions

Digest partitions route the incoming data while it is "in progress".

For more information, read the Digest Rules documentation.

Warning

Do not POST to this API unless the cluster is running fine, with all members connected and active. All digests stop for a few seconds when being applied.

Digest does not start before all nodes are ready, thus if a node is failing, digest does not resume.

Show digest partition information

GET/POST the setting to hand-edit where each partition goes. You cannot reduce the number of partitions.

Description Returns a list of the digest partitions.
MethodGET /api/v1/clusterconfig/ingestpartitions
Authentication Requiredyes
Return Codes
200 Request complete
400 Bad authentication
500 Request failed
http
GET /api/v1/clusterconfig/ingestpartitions

To redirect the output to a JSON file, run the script like this:

shell
$ script>$YOUR_FILE_NAME.json

You can then use this as the payload in further requests when updating the information.

Mac OS or Linux (curl)
shell
curl -v -X GET $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json"
Mac OS or Linux (curl) One-line
shell
curl -v -X GET $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json"
Windows Cmd and curl
shell
curl -v -X GET $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions ^
    -H "Authorization: Bearer $TOKEN" ^
    -H "Content-Type: application/json"
Windows Powershell and curl
powershell
curl.exe -X GET 
    -H "Authorization: Bearer $TOKEN"
    -H "Content-Type: application/json"
    "$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions"
Perl
perl
#!/usr/bin/perl

use HTTP::Request;
use LWP;

my $INGEST_TOKEN = "TOKEN";

my $uri = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions';

my $req = HTTP::Request->new("GET", $uri );

$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");


my $lwp = LWP::UserAgent->new;

my $result = $lwp->request( $req );

print $result->{"_content"},"\n";
Python
python
#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions'

resp = requests.get(url,
                     headers = {
   "Authorization" : "Bearer $TOKEN",
   "Content-Type" : "application/json"
}
)

print(resp.text)
Node.js
javascript
const https = require('https');

let request = https.get('$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions', (res) => {
  if (res.statusCode !== 200) {
    console.error(`Error from server. Code: ${res.statusCode}`);
    res.resume();
    return;
  }

  let data = '';

  res.on('data', (chunk) => {
    data += chunk;
  });

  res.on('close', () => {
    console.log('Response:');
    console.log(JSON.parse(data));
  });
});

    return(undef,undef);
}

Update digest partition settings

Description Manage the digest partitions to control manually where each partition is.
MethodPOST /api/v1/clusterconfig/ingestpartitions
Request Data 
Authentication Requiredyes
Return Codes
200 Request complete
400 Bad authentication
500 Request failed
http
POST /api/v1/clusterconfig/ingestpartitions
Mac OS or Linux (curl)
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '@$YOUR_FILE_NAME.json'
Mac OS or Linux (curl) One-line
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '@$YOUR_FILE_NAME.json'
Windows Cmd and curl
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions ^
    -H "Authorization: Bearer $TOKEN" ^
    -H "Content-Type: application/json" ^
    -d '@$YOUR_FILE_NAME.json'
Windows Powershell and curl
powershell
curl.exe -X POST 
    -H "Authorization: Bearer $TOKEN"
    -H "Content-Type: application/json"
    -d '@$YOUR_FILE_NAME.json'
    "$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions"
Perl
perl
#!/usr/bin/perl

use HTTP::Request;
use LWP;

my $INGEST_TOKEN = "TOKEN";

my $uri = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions';

my $json = '@$YOUR_FILE_NAME.json';
my $req = HTTP::Request->new("POST", $uri );

$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");

$req->content( $json );

my $lwp = LWP::UserAgent->new;

my $result = $lwp->request( $req );

print $result->{"_content"},"\n";
Python
python
#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions'
mydata = r'''@$YOUR_FILE_NAME.json'''

resp = requests.post(url,
                     data = mydata,
                     headers = {
   "Authorization" : "Bearer $TOKEN",
   "Content-Type" : "application/json"
}
)

print(resp.text)
Node.js
javascript
const https = require('https');

const data = JSON.stringify(
    @$YOUR_FILE_NAME.json
);


const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();

Update digest partition settings using defaults

Invoke setdefaults to distribute the current number of partitions evenly among the known nodes in the cluster

Description Control partition distribution to nodes in the cluster.
MethodPOST /api/v1/clusterconfig/ingestpartitions
Request Data 
Authentication Requiredyes
Return Codes
200 Request complete
400 Bad authentication
500 Request failed
http
POST /api/v1/clusterconfig/ingestpartitions/setdefaults
Mac OS or Linux (curl)
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '@$YOUR_FILE_NAME.json'
Mac OS or Linux (curl) One-line
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '@$YOUR_FILE_NAME.json'
Windows Cmd and curl
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults ^
    -H "Authorization: Bearer $TOKEN" ^
    -H "Content-Type: application/json" ^
    -d '@$YOUR_FILE_NAME.json'
Windows Powershell and curl
powershell
curl.exe -X POST 
    -H "Authorization: Bearer $TOKEN"
    -H "Content-Type: application/json"
    -d '@$YOUR_FILE_NAME.json'
    "$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults"
Perl
perl
#!/usr/bin/perl

use HTTP::Request;
use LWP;

my $INGEST_TOKEN = "TOKEN";

my $uri = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults';

my $json = '@$YOUR_FILE_NAME.json';
my $req = HTTP::Request->new("POST", $uri );

$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");

$req->content( $json );

my $lwp = LWP::UserAgent->new;

my $result = $lwp->request( $req );

print $result->{"_content"},"\n";
Python
python
#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults'
mydata = r'''@$YOUR_FILE_NAME.json'''

resp = requests.post(url,
                     data = mydata,
                     headers = {
   "Authorization" : "Bearer $TOKEN",
   "Content-Type" : "application/json"
}
)

print(resp.text)
Node.js
javascript
const https = require('https');

const data = JSON.stringify(
    @$YOUR_FILE_NAME.json
);


const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/setdefaults',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();

Manage digest partition settings from node

Invoke distribute-evenly-from-host to reassign partitions currently assigned to $NODE_ID to the other nodes in the cluster.

Description Move partitions between nodes.  
MethodPOST /api/v1/clusterconfig/ingestpartitions/id 
Request Data  
Authentication Requiredyes 
Path ArgumentsDescriptionData typeRequired?
id ID of the node configuration stringrequired
Return Codes 
200 Request complete  
400 Bad authentication  
500 Request failed  
http
POST /api/v1/clusterconfig/ingestpartitions/distribute-evenly-from-host/$NODE_ID
Mac OS or Linux (curl)
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/distribute-evenly-from-host/$NODE_ID \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '@$YOUR_FILE_NAME.json'
Mac OS or Linux (curl) One-line
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/distribute-evenly-from-host/$NODE_ID \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '@$YOUR_FILE_NAME.json'
Windows Cmd and curl
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/distribute-evenly-from-host/$NODE_ID ^
    -H "Authorization: Bearer $TOKEN" ^
    -H "Content-Type: application/json" ^
    -d '@$YOUR_FILE_NAME.json'
Windows Powershell and curl
powershell
curl.exe -X POST 
    -H "Authorization: Bearer $TOKEN"
    -H "Content-Type: application/json"
    -d '@$YOUR_FILE_NAME.json'
    "$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/distribute-evenly-from-host/$NODE_ID"
Perl
perl
#!/usr/bin/perl

use HTTP::Request;
use LWP;

my $INGEST_TOKEN = "TOKEN";

my $uri = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/distribute-evenly-from-host/$NODE_ID';

my $json = '@$YOUR_FILE_NAME.json';
my $req = HTTP::Request->new("POST", $uri );

$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");

$req->content( $json );

my $lwp = LWP::UserAgent->new;

my $result = $lwp->request( $req );

print $result->{"_content"},"\n";
Python
python
#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/distribute-evenly-from-host/$NODE_ID'
mydata = r'''@$YOUR_FILE_NAME.json'''

resp = requests.post(url,
                     data = mydata,
                     headers = {
   "Authorization" : "Bearer $TOKEN",
   "Content-Type" : "application/json"
}
)

print(resp.text)
Node.js
javascript
const https = require('https');

const data = JSON.stringify(
    @$YOUR_FILE_NAME.json
);


const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/clusterconfig/ingestpartitions/distribute-evenly-from-host/$NODE_ID',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();