Lookup API

Use the lookup endpoints to download, upload, and delete files that can be used by the match() function.

You can upload files in CSV or JSON format. Upload files as multipart form data. The file should be in a part named file.

Files can be uploaded to a repository or files can be uploaded as shared files that are available in all repositories. Uploading a file with the same name will replace the original file that was uploaded previously. Shared files are visible in all repositories and can be used by all users. Only root users can upload, delete and edit shared files. Shared files are referenced from functions like match(), by filename preprended with /shared/. For example /shared/myfile.csv

Download Files

LogScale allows you to download files that have been uploaded to it. For the GET form, this endpoint returns the document.

There is also a GraphQL query for this.

Description	Download a file from a repository in the cluster.
Method	`GET /api/v1/repositories/repository/files/filename`
Authentication Required	yes
Path Arguments	Description	Data type	Required?
`filename`	Name of the file to download; must be .csv or .json.	`string`	required
`repository`	The repository name	`string`	required
Return Codes
200	Request complete
400	Bad request
500	Request failed

http

GET /api/v1/repositories/repository/location/files/filename

Mac OS or Linux (curl)

shell

curl -v -X GET $YOUR_LOGSCALE_URL/api/v1/repositories/repository/location/files/$YOUR_FILE_NAME.json \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json"

Mac OS or Linux (curl) One-line

shell

curl -v -X GET $YOUR_LOGSCALE_URL/api/v1/repositories/repository/location/files/$YOUR_FILE_NAME.json \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json"

Windows Cmd and curl

shell

curl -v -X GET $YOUR_LOGSCALE_URL/api/v1/repositories/repository/location/files/$YOUR_FILE_NAME.json ^
    -H "Authorization: Bearer $TOKEN" ^
    -H "Content-Type: application/json"

Windows Powershell and curl

powershell

curl.exe -X GET 
    -H "Authorization: Bearer $TOKEN"
    -H "Content-Type: application/json"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;

my $TOKEN = "TOKEN";

my $uri = '$YOUR_LOGSCALE_URL/api/v1/repositories/repository/location/files/$YOUR_FILE_NAME.json';

my $req = HTTP::Request->new("GET", $uri );

$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");


my $lwp = LWP::UserAgent->new;

my $result = $lwp->request( $req );

print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/repositories/repository/location/files/$YOUR_FILE_NAME.json'

resp = requests.get(url,
                     headers = {
   "Authorization" : "Bearer $TOKEN",
   "Content-Type" : "application/json"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');

let request = https.get('$YOUR_LOGSCALE_URL/api/v1/repositories/repository/location/files/$YOUR_FILE_NAME.json', (res) => {
  if (res.statusCode !== 200) {
    console.error(`Error from server. Code: ${res.statusCode}`);
    res.resume();
    return;
  }

  let data = '';

  res.on('data', (chunk) => {
    data += chunk;
  });

  res.on('close', () => {
    console.log('Response:');
    console.log(JSON.parse(data));
  });
});

    return(undef,undef);
}

Download Shared Files

LogScale allows you to download shared files that have been uploaded to it. For the GET form, this endpoint returns the document.

Description	Download a shared file in the cluster.
Method	`GET /api/v1/uploadedfiles/shared/filename`
Authentication Required	yes
Path Arguments	Description	Data type	Required?
`filename`	Name of the file to download; must be .csv or .json.	`string`	required
Return Codes
200	Request complete
400	Bad request
500	Request failed

http

GET /api/v1/uploadedfiles/shared/filename

Mac OS or Linux (curl)

shell

curl -v -X GET $YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared/$YOUR_FILE_NAME.json \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json"

Mac OS or Linux (curl) One-line

shell

curl -v -X GET $YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared/$YOUR_FILE_NAME.json \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json"

Windows Cmd and curl

shell

curl -v -X GET $YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared/$YOUR_FILE_NAME.json ^
    -H "Authorization: Bearer $TOKEN" ^
    -H "Content-Type: application/json"

Windows Powershell and curl

powershell

curl.exe -X GET 
    -H "Authorization: Bearer $TOKEN"
    -H "Content-Type: application/json"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;

my $TOKEN = "TOKEN";

my $uri = '$YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared/$YOUR_FILE_NAME.json';

my $req = HTTP::Request->new("GET", $uri );

$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");


my $lwp = LWP::UserAgent->new;

my $result = $lwp->request( $req );

print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared/$YOUR_FILE_NAME.json'

resp = requests.get(url,
                     headers = {
   "Authorization" : "Bearer $TOKEN",
   "Content-Type" : "application/json"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');

let request = https.get('$YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared/$YOUR_FILE_NAME.json', (res) => {
  if (res.statusCode !== 200) {
    console.error(`Error from server. Code: ${res.statusCode}`);
    res.resume();
    return;
  }

  let data = '';

  res.on('data', (chunk) => {
    data += chunk;
  });

  res.on('close', () => {
    console.log('Response:');
    console.log(JSON.parse(data));
  });
});

    return(undef,undef);
}

Upload Files

LogScale allows you to upload files. The files must be JSON or CSV files.

Description	Upload a file to a repository.
Method	`POST /api/v1/repositories/repository/files/filename`
Request Data
Authentication Required	yes
Path Arguments	Description	Data type	Required?
`filename`	Name of the file to upload; must be .csv or .json.	`string`	required
`repository`	The repository name	`string`	required
Return Codes
200	Request complete
400	Bad request
500	Request failed

http

POST api/v1/repositories/$REPOSITORY_NAME/files/filename

Mac OS or Linux (curl)

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/files \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
-F "file=@$YOUR_FILE_NAME.json

Mac OS or Linux (curl) One-line

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/files \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
-F "file=@$YOUR_FILE_NAME.json

Windows Cmd and curl

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/files ^
    -H "Authorization: Bearer $TOKEN" ^
    -H "Content-Type: application/json" ^
-F "file=@$YOUR_FILE_NAME.json

Windows Powershell and curl

powershell

curl.exe -X POST 
    -H "Authorization: Bearer $TOKEN"
    -H "Content-Type: application/json"
-F "file=@$YOUR_FILE_NAME.json
"$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/files"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;

my $TOKEN = "TOKEN";
my %form = (
    "file" => "@$YOUR_FILE_NAME.json",
);

my $uri = '$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/files';

my $json = '';
my $req = HTTP::Request->new("POST", $uri, \%form );

$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");

$req->content( $json );

my $lwp = LWP::UserAgent->new;

my $result = $lwp->request( $req );

print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/files'
mydata = r''

files = {
'file': (None,'@$YOUR_FILE_NAME.json'),
}
resp = requests.post(url,
                     files = files,
                     headers = {
   "Authorization" : "Bearer $TOKEN",
   "Content-Type" : "application/json"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');



const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/files',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();

Upload Shared Files

If you are a self-hosted user, LogScale allows you to upload shared files. The files must be JSON or CSV files.

Description	Upload a shared file to a repository.
Method	`POST /api/v1/uploadedfiles/shared/filename`
Request Data
Authentication Required	yes
Path Arguments	Description	Data type	Required?
`filename`	Name of the file to upload; must be .csv or .json.	`string`	required
Return Codes
200	Request complete
400	Bad request
500	Request failed

http

POST /api/v1/uploadedfiles/shared/filename

Mac OS or Linux (curl)

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
-F "file=@$YOUR_FILE_NAME.json

Mac OS or Linux (curl) One-line

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
-F "file=@$YOUR_FILE_NAME.json

Windows Cmd and curl

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared ^
    -H "Authorization: Bearer $TOKEN" ^
    -H "Content-Type: application/json" ^
-F "file=@$YOUR_FILE_NAME.json

Windows Powershell and curl

powershell

curl.exe -X POST 
    -H "Authorization: Bearer $TOKEN"
    -H "Content-Type: application/json"
-F "file=@$YOUR_FILE_NAME.json
"$YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;

my $TOKEN = "TOKEN";
my %form = (
    "file" => "@$YOUR_FILE_NAME.json",
);

my $uri = '$YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared';

my $json = '';
my $req = HTTP::Request->new("POST", $uri, \%form );

$req->header("Authorization" => "Bearer $TOKEN");
$req->header("Content-Type" => "application/json");

$req->content( $json );

my $lwp = LWP::UserAgent->new;

my $result = $lwp->request( $req );

print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared'
mydata = r''

files = {
'file': (None,'@$YOUR_FILE_NAME.json'),
}
resp = requests.post(url,
                     files = files,
                     headers = {
   "Authorization" : "Bearer $TOKEN",
   "Content-Type" : "application/json"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');



const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();

Delete Files

LogScale allows you to delete files that have been uploaded to it. For Cloud users, a file can be deleted from all repositories using the following:

Description	Delete a file from a repository.
Method	`DELETE /api/v1/repositories/repository/files/filename`
Authentication Required	yes
Path Arguments	Description	Data type	Required?
`filename`	Name of the file to download; must be .csv or .json.	`string`	required
`repository`	The repository name	`string`	required
Return Codes
200	Request complete
400	Bad request
500	Request failed

http

DELETE /api/v1/repositories/REPOSITORY_NAME/files/filename

Mac OS or Linux (curl)

shell

curl -v -X DELETE $YOUR_LOGSCALE_URL/api/v1/repositories/REPOSITORY_NAME/files/$YOUR_FILE_NAME.json \
    -H "Authorization: Bearer $TOKEN"

Mac OS or Linux (curl) One-line

shell

curl -v -X DELETE $YOUR_LOGSCALE_URL/api/v1/repositories/REPOSITORY_NAME/files/$YOUR_FILE_NAME.json \
    -H "Authorization: Bearer $TOKEN"

Windows Cmd and curl

shell

curl -v -X DELETE $YOUR_LOGSCALE_URL/api/v1/repositories/REPOSITORY_NAME/files/$YOUR_FILE_NAME.json ^
    -H "Authorization: Bearer $TOKEN"

Windows Powershell and curl

powershell

curl.exe -X DELETE 
    -H "Authorization: Bearer $TOKEN"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;

my $TOKEN = "TOKEN";

my $uri = '$YOUR_LOGSCALE_URL/api/v1/repositories/REPOSITORY_NAME/files/$YOUR_FILE_NAME.json';

my $req = HTTP::Request->new("DELETE", $uri );

$req->header("Authorization" => "Bearer $TOKEN");


my $lwp = LWP::UserAgent->new;

my $result = $lwp->request( $req );

print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/repositories/REPOSITORY_NAME/files/$YOUR_FILE_NAME.json'

resp = requests.delete(url,
                     headers = {
   "Authorization" : "Bearer $TOKEN"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');

let request = https.delete('$YOUR_LOGSCALE_URL/api/v1/repositories/REPOSITORY_NAME/files/$YOUR_FILE_NAME.json', (res) => {
  if (res.statusCode !== 200) {
    console.error(`Error from server. Code: ${res.statusCode}`);
    res.resume();
    return;
  }

  let data = '';

  res.on('data', (chunk) => {
    data += chunk;
  });

  res.on('close', () => {
    console.log('Response:');
    console.log(JSON.parse(data));
  });
});

    return(undef,undef);
}

Delete Shared Files

LogScale allows you to delete shared files that have been uploaded to it. For self-hosted users, a file can be deleted from all repositories using the following:

Description	Delete a shared file in a cluster.
Method	`DELETE /api/v1/uploadedfiles/shared/filename`
Authentication Required	yes
Path Arguments	Description	Data type	Required?
`filename`	Name of the file to download; must be .csv or .json.	`string`	required
Return Codes
200	Request complete
400	Bad request
500	Request failed

http

DELETE /api/v1/uploadedfiles/shared/filename

Mac OS or Linux (curl)

shell

curl -v -X DELETE $YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared/$YOUR_FILE_NAME.json \
    -H "Authorization: Bearer $TOKEN"

Mac OS or Linux (curl) One-line

shell

curl -v -X DELETE $YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared/$YOUR_FILE_NAME.json \
    -H "Authorization: Bearer $TOKEN"

Windows Cmd and curl

shell

curl -v -X DELETE $YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared/$YOUR_FILE_NAME.json ^
    -H "Authorization: Bearer $TOKEN"

Windows Powershell and curl

powershell

curl.exe -X DELETE 
    -H "Authorization: Bearer $TOKEN"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;

my $TOKEN = "TOKEN";

my $uri = '$YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared/$YOUR_FILE_NAME.json';

my $req = HTTP::Request->new("DELETE", $uri );

$req->header("Authorization" => "Bearer $TOKEN");


my $lwp = LWP::UserAgent->new;

my $result = $lwp->request( $req );

print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared/$YOUR_FILE_NAME.json'

resp = requests.delete(url,
                     headers = {
   "Authorization" : "Bearer $TOKEN"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');

let request = https.delete('$YOUR_LOGSCALE_URL/api/v1/uploadedfiles/shared/$YOUR_FILE_NAME.json', (res) => {
  if (res.statusCode !== 200) {
    console.error(`Error from server. Code: ${res.statusCode}`);
    res.resume();
    return;
  }

  let data = '';

  res.on('data', (chunk) => {
    data += chunk;
  });

  res.on('close', () => {
    console.log('Response:');
    console.log(JSON.parse(data));
  });
});

    return(undef,undef);
}

Example Contents for a CSV File

Whitespace gets included in the keys and values. To include the separator "," in a value, quote using the " character.

csv

userid,name
1,chr
2,krab
"4","p,m"
7,mgr

Example: contents for a file in JSON format using an object as root of the file.

In this variant, the key field does not have a name.

json

{
  "1": { "name": "chr" },
  "2": { "name": "krab" },
  "4": { "name": "pmm" },
  "7": { "name": "mgr" }
}

Example: contents for a file in JSON format using an array as root of the file.

In this variant, you select which field is the key using the field parameter in match().

json

[
  { "userid": "1", "name": "chr" },
  { "userid": "2", "name": "krab" },
  { "userid": "4", "name": "pmm" },
  { "userid": "7", "name": "mgr" }
]

APIs

API Authentication

Cluster Management API

Health Check API

Ingest API

Lookup API

Redact Events API

Search API

Software Libraries

Lookup API

Download Files

Download Shared Files

Upload Files

Upload Shared Files

Delete Files

Delete Shared Files

Example Contents for a CSV File

Example: contents for a file in JSON format using an object as root of the file.

Example: contents for a file in JSON format using an array as root of the file.

Enter search term