HTTP Event Collector (HEC)
LogScale's HEC API is an alternative HTTP ingest endpoint. You will find
it at /api/v1/ingest/hec
and at /services/collector
.
Note
The best practices described in Ingest via API Best Practices also apply to the HEC API.
Format of Data
Ingested data is a series of whitespace delimited JSON objects, containing one or more of the following elements. All elements are optional.
Member | Description |
---|---|
time
|
Time in seconds since January 1, 1970 in UTC. This can be a integer or floating point number to support milliseconds. LogScale represents time with millisecond precision. Times are interpreted as follows:
|
timezone
|
Can be used to describe the time zone in which the event
happened. Defaults to
Z (i.e., UTC).
|
index
| Optional name of the repository to ingest into. In public-facing API's this must — if present — be equal to the repository used to create the ingest token used for authentication. In private cluster setups, humio can be configured to allow these to be different. See below. |
sourcetype
|
Translated to #type inside LogScale. If
set, this is used to choose which LogScale parser to use for
extracting fields.
To refer to a parser within the given repository just the name
can be provided. To refer to the parser from a package
installed into the repository, use the form
|
source
| Translated to the @source field in LogScale. Typically used to designate the path to the file that is being shipped to LogScale. |
host
| Translated to the @host field in LogScale. Typically used to designate the origin host. |
event
|
This can JSON Object, a String, array or object. Translated to
the @rawstring field in LogScale. When this
is a JSON Object, all members of the object will become
accessible fields in humio with no further processing. If it is
a string, the key/value parser is always applied to the string
to extract elements. The key/value parser searches for
key=value ,
key="value" or
key='value' .
|
fields
|
JSON object containing extra fields to the event. This can be
used if event is a string,
boolean or number and it is pre-processed prior to ingest to
extract fields. Tags #tags can be added to
the event by specifying fields starting with
# . It cannot be used if
event is an array or an
object.
|
Data Handling for Large Data Events
The following examples detail the resulting ingested data for different raw event volumes. For more information on limits, see Limits & Standards.
Ingest Data within Limits
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
-H "Authorization: Bearer $INGEST_TOKEN" \
-H "Content-Type: text/plain; charset=utf-8" \
-d @- << EOF
{
"fields": {
"#someTag": "a-small-value",
"someField1": "tiny-data-value",
"someField2": "small-data-value"
},
"event": "also-a-tiny-data-value"
}
EOF
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
-H "Authorization: Bearer $INGEST_TOKEN" \
-H "Content-Type: text/plain; charset=utf-8" \
-d @- << EOF
{
"fields": {
"#someTag": "a-small-value",
"someField1": "tiny-data-value",
"someField2": "small-data-value"
},
"event": "also-a-tiny-data-value"
}
EOF
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec ^
-H "Authorization: Bearer $INGEST_TOKEN" ^
-H "Content-Type: text/plain; charset=utf-8" ^
-d @'{ ^
"fields": { ^
"#someTag": "a-small-value", ^
"someField1": "tiny-data-value", ^
"someField2": "small-data-value" ^
}, ^
"event": "also-a-tiny-data-value" ^
} '
curl.exe -X POST
-H "Authorization: Bearer $INGEST_TOKEN"
-H "Content-Type: text/plain; charset=utf-8"
-d '{
"fields": {
"#someTag": "a-small-value",
"someField1": "tiny-data-value",
"someField2": "small-data-value"
},
"event": "also-a-tiny-data-value"
}'
"$YOUR_LOGSCALE_URL/api/v1/ingest/hec"
#!/usr/bin/perl
use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec';
my $json = '{
"fields": {
"#someTag": "a-small-value",
"someField1": "tiny-data-value",
"someField2": "small-data-value"
},
"event": "also-a-tiny-data-value"
}';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Authorization" => "Bearer $INGEST_TOKEN");
$req->header("Content-Type" => "text/plain; charset=utf-8");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";
#! /usr/local/bin/python3
import requests
url = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec'
mydata = r'''{
"fields": {
"#someTag": "a-small-value",
"someField1": "tiny-data-value",
"someField2": "small-data-value"
},
"event": "also-a-tiny-data-value"
}'''
resp = requests.post(url,
data = mydata,
headers = {
"Authorization" : "Bearer $INGEST_TOKEN",
"Content-Type" : "text/plain; charset=utf-8"
}
)
print(resp.text)
const https = require('https');
const data = JSON.stringify(
{
"fields": {
"#someTag": "a-small-value",
"someField1": "tiny-data-value",
"someField2": "small-data-value"
},
"event": "also-a-tiny-data-value"
}
);
const options = {
hostname: '$YOUR_LOGSCALE_URL/api/v1/ingest/hec',
path: '/graphql',
port: 443,
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': data.length,
Authorization: 'BEARER ' + process.env.TOKEN,
'User-Agent': 'Node',
},
};
const req = https.request(options, (res) => {
let data = '';
console.log(`statusCode: ${res.statusCode}`);
res.on('data', (d) => {
data += d;
});
res.on('end', () => {
console.log(JSON.parse(data).data);
});
});
req.on('error', (error) => {
console.error(error);
});
req.write(data);
req.end();
Field | Value |
---|---|
#repo | developer |
#type | none |
@id | L4tQKldzrldCwMCVc8PIq7Wp_0_0_1719304163 |
@ingesttimestamp | 1719304163717 |
@rawstring | also-a-tiny-data-value |
@timestamp | 1719304163717 |
@timestamp.nanos | 0 |
@timezone | Z |
#someTag | a-small-value |
someField1 | tiny-data-value |
someField2 | small-data-value |
All expected data and fields are present, and they are not truncated.
Raw String is Too Large
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
-H "Authorization: Bearer $INGEST_TOKEN" \
-H "Content-Type: text/plain; charset=utf-8" \
-d @- << EOF
{
"fields": {
"#someTag": "a-small-value",
"someField1": "someValue1",
"someField2": "someValue2"
},
"event": "... More than 1MB of data ..."
}
EOF
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
-H "Authorization: Bearer $INGEST_TOKEN" \
-H "Content-Type: text/plain; charset=utf-8" \
-d @- << EOF
{
"fields": {
"#someTag": "a-small-value",
"someField1": "someValue1",
"someField2": "someValue2"
},
"event": "... More than 1MB of data ..."
}
EOF
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec ^
-H "Authorization: Bearer $INGEST_TOKEN" ^
-H "Content-Type: text/plain; charset=utf-8" ^
-d @'{ ^
"fields": { ^
"#someTag": "a-small-value", ^
"someField1": "someValue1", ^
"someField2": "someValue2" ^
}, ^
"event": "... More than 1MB of data ..." ^
} '
curl.exe -X POST
-H "Authorization: Bearer $INGEST_TOKEN"
-H "Content-Type: text/plain; charset=utf-8"
-d '{
"fields": {
"#someTag": "a-small-value",
"someField1": "someValue1",
"someField2": "someValue2"
},
"event": "... More than 1MB of data ..."
}'
"$YOUR_LOGSCALE_URL/api/v1/ingest/hec"
#!/usr/bin/perl
use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec';
my $json = '{
"fields": {
"#someTag": "a-small-value",
"someField1": "someValue1",
"someField2": "someValue2"
},
"event": "... More than 1MB of data ..."
}';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Authorization" => "Bearer $INGEST_TOKEN");
$req->header("Content-Type" => "text/plain; charset=utf-8");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";
#! /usr/local/bin/python3
import requests
url = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec'
mydata = r'''{
"fields": {
"#someTag": "a-small-value",
"someField1": "someValue1",
"someField2": "someValue2"
},
"event": "... More than 1MB of data ..."
}'''
resp = requests.post(url,
data = mydata,
headers = {
"Authorization" : "Bearer $INGEST_TOKEN",
"Content-Type" : "text/plain; charset=utf-8"
}
)
print(resp.text)
const https = require('https');
const data = JSON.stringify(
{
"fields": {
"#someTag": "a-small-value",
"someField1": "someValue1",
"someField2": "someValue2"
},
"event": "... More than 1MB of data ..."
}
);
const options = {
hostname: '$YOUR_LOGSCALE_URL/api/v1/ingest/hec',
path: '/graphql',
port: 443,
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': data.length,
Authorization: 'BEARER ' + process.env.TOKEN,
'User-Agent': 'Node',
},
};
const req = https.request(options, (res) => {
let data = '';
console.log(`statusCode: ${res.statusCode}`);
res.on('data', (d) => {
data += d;
});
res.on('end', () => {
console.log(JSON.parse(data).data);
});
});
req.on('error', (error) => {
console.error(error);
});
req.write(data);
req.end();
Field | Value |
---|---|
#error | true |
#repo | developer |
#type | json |
@error | true |
@error_msg | Input too big. Event truncated at 1048576 |
@error_msg[0] | Input too big. Event truncated at 1048576 |
@id | GveuGB8hexKWpraoahisoA46_0_0_1719301041 |
@ingesttimestamp | 1719301041888 |
@input_size | 1048677 |
@rawstring | Truncated data |
@timestamp | 1719301041888 |
@timestamp.nanos | 0 |
@timezone | Z |
#someTag | a-small-value |
The someField1 and someField2 are not present, but #someTag is.
Field is Too Large
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
-H "Authorization: Bearer $INGEST_TOKEN" \
-H "Content-Type: text/plain; charset=utf-8" \
-d @- << EOF
{
"fields": {
"#someTag": "a-small-value",
"someField1": "tiny-data-value",
"someField2": "... More than 1MB of data ..."
},
"event": "also-a-tiny-data-value"
}
EOF
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
-H "Authorization: Bearer $INGEST_TOKEN" \
-H "Content-Type: text/plain; charset=utf-8" \
-d @- << EOF
{
"fields": {
"#someTag": "a-small-value",
"someField1": "tiny-data-value",
"someField2": "... More than 1MB of data ..."
},
"event": "also-a-tiny-data-value"
}
EOF
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec ^
-H "Authorization: Bearer $INGEST_TOKEN" ^
-H "Content-Type: text/plain; charset=utf-8" ^
-d @'{ ^
"fields": { ^
"#someTag": "a-small-value", ^
"someField1": "tiny-data-value", ^
"someField2": "... More than 1MB of data ..." ^
}, ^
"event": "also-a-tiny-data-value" ^
} '
curl.exe -X POST
-H "Authorization: Bearer $INGEST_TOKEN"
-H "Content-Type: text/plain; charset=utf-8"
-d '{
"fields": {
"#someTag": "a-small-value",
"someField1": "tiny-data-value",
"someField2": "... More than 1MB of data ..."
},
"event": "also-a-tiny-data-value"
}'
"$YOUR_LOGSCALE_URL/api/v1/ingest/hec"
#!/usr/bin/perl
use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec';
my $json = '{
"fields": {
"#someTag": "a-small-value",
"someField1": "tiny-data-value",
"someField2": "... More than 1MB of data ..."
},
"event": "also-a-tiny-data-value"
}';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Authorization" => "Bearer $INGEST_TOKEN");
$req->header("Content-Type" => "text/plain; charset=utf-8");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";
#! /usr/local/bin/python3
import requests
url = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec'
mydata = r'''{
"fields": {
"#someTag": "a-small-value",
"someField1": "tiny-data-value",
"someField2": "... More than 1MB of data ..."
},
"event": "also-a-tiny-data-value"
}'''
resp = requests.post(url,
data = mydata,
headers = {
"Authorization" : "Bearer $INGEST_TOKEN",
"Content-Type" : "text/plain; charset=utf-8"
}
)
print(resp.text)
const https = require('https');
const data = JSON.stringify(
{
"fields": {
"#someTag": "a-small-value",
"someField1": "tiny-data-value",
"someField2": "... More than 1MB of data ..."
},
"event": "also-a-tiny-data-value"
}
);
const options = {
hostname: '$YOUR_LOGSCALE_URL/api/v1/ingest/hec',
path: '/graphql',
port: 443,
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': data.length,
Authorization: 'BEARER ' + process.env.TOKEN,
'User-Agent': 'Node',
},
};
const req = https.request(options, (res) => {
let data = '';
console.log(`statusCode: ${res.statusCode}`);
res.on('data', (d) => {
data += d;
});
res.on('end', () => {
console.log(JSON.parse(data).data);
});
});
req.on('error', (error) => {
console.error(error);
});
req.write(data);
req.end();
Field | Value |
---|---|
#error | true |
#repo | developer |
#type | json |
@error | true |
@error_msg | Input too big. Event truncated at 1048576 |
@error_msg[0] | Input too big. Event truncated at 1048576 |
@id | GveuGB8hexKWpraoahisoA46_1_1_1719303197 |
@ingesttimestamp | 1719303197844 |
@input_size | 1049579 |
@rawstring | also-a-tiny-data-value |
@timestamp | 1719303197844 |
@timestamp.nanos | 0 |
@timezone | Z |
#someTag | a-small-value |
The someField1 and someField2 are not present, but #someTag is.
Tag is Too Large
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
-H "Authorization: Bearer $INGEST_TOKEN" \
-H "Content-Type: text/plain; charset=utf-8" \
-d @- << EOF
{
"fields": {
"#someTag": "... More than 65535 bytes of data ...",
"someField1": "tiny-data-value",
"someField2": "small-data-value"
},
"event": "also-a-tiny-data-value"
}
EOF
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
-H "Authorization: Bearer $INGEST_TOKEN" \
-H "Content-Type: text/plain; charset=utf-8" \
-d @- << EOF
{
"fields": {
"#someTag": "... More than 65535 bytes of data ...",
"someField1": "tiny-data-value",
"someField2": "small-data-value"
},
"event": "also-a-tiny-data-value"
}
EOF
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec ^
-H "Authorization: Bearer $INGEST_TOKEN" ^
-H "Content-Type: text/plain; charset=utf-8" ^
-d @'{ ^
"fields": { ^
"#someTag": "... More than 65535 bytes of data ...", ^
"someField1": "tiny-data-value", ^
"someField2": "small-data-value" ^
}, ^
"event": "also-a-tiny-data-value" ^
} '
curl.exe -X POST
-H "Authorization: Bearer $INGEST_TOKEN"
-H "Content-Type: text/plain; charset=utf-8"
-d '{
"fields": {
"#someTag": "... More than 65535 bytes of data ...",
"someField1": "tiny-data-value",
"someField2": "small-data-value"
},
"event": "also-a-tiny-data-value"
}'
"$YOUR_LOGSCALE_URL/api/v1/ingest/hec"
#!/usr/bin/perl
use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec';
my $json = '{
"fields": {
"#someTag": "... More than 65535 bytes of data ...",
"someField1": "tiny-data-value",
"someField2": "small-data-value"
},
"event": "also-a-tiny-data-value"
}';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Authorization" => "Bearer $INGEST_TOKEN");
$req->header("Content-Type" => "text/plain; charset=utf-8");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";
#! /usr/local/bin/python3
import requests
url = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec'
mydata = r'''{
"fields": {
"#someTag": "... More than 65535 bytes of data ...",
"someField1": "tiny-data-value",
"someField2": "small-data-value"
},
"event": "also-a-tiny-data-value"
}'''
resp = requests.post(url,
data = mydata,
headers = {
"Authorization" : "Bearer $INGEST_TOKEN",
"Content-Type" : "text/plain; charset=utf-8"
}
)
print(resp.text)
const https = require('https');
const data = JSON.stringify(
{
"fields": {
"#someTag": "... More than 65535 bytes of data ...",
"someField1": "tiny-data-value",
"someField2": "small-data-value"
},
"event": "also-a-tiny-data-value"
}
);
const options = {
hostname: '$YOUR_LOGSCALE_URL/api/v1/ingest/hec',
path: '/graphql',
port: 443,
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': data.length,
Authorization: 'BEARER ' + process.env.TOKEN,
'User-Agent': 'Node',
},
};
const req = https.request(options, (res) => {
let data = '';
console.log(`statusCode: ${res.statusCode}`);
res.on('data', (d) => {
data += d;
});
res.on('end', () => {
console.log(JSON.parse(data).data);
});
});
req.on('error', (error) => {
console.error(error);
});
req.write(data);
req.end();
Event data is too large
Entire request is dropped, no data in LogScale.
Authentication
You will need to provide a Ingest Tokens in
the HTTP Authorization
header.
The ingest token contains the name of the repository the data is stored in, and ingested events will be stored in the repository corresponding to the ingest token.
If using an Organization API Token with the
Ingest across all repositories within organization
permission, then HEC allows ingest to any
repository specified as "index":
"repository-name"
in the body of a message, as long as the ingest
token is valid for any repository on the LogScale cluster. If the named
repository does not exist then an error will be returned.
This is a potential security issue on a public API endpoint, so this option should only be used inside a trusted environment.
Example
Below is an example of the sending a JSON document for ingest:
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
-H "Authorization: Bearer $INGEST_TOKEN" \
-H "Content-Type: text/plain; charset=utf-8" \
-d @- << EOF
{
"time" : 1537537729.0,
"event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
"source" : "/var/log/application.log",
"sourcetype" : "applog",
"fields" : { "#env" : "prod" }
}
{
"time" : 1537535729.0,
"event" : {
"message" : "System shutdown",
"host" : { "ip" : "127.0.0.1", "port" : 2222 }
},
"fields" : { "#datacenter" : "amazon-east1" }
}
EOF
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
-H "Authorization: Bearer $INGEST_TOKEN" \
-H "Content-Type: text/plain; charset=utf-8" \
-d @- << EOF
{
"time" : 1537537729.0,
"event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
"source" : "/var/log/application.log",
"sourcetype" : "applog",
"fields" : { "#env" : "prod" }
}
{
"time" : 1537535729.0,
"event" : {
"message" : "System shutdown",
"host" : { "ip" : "127.0.0.1", "port" : 2222 }
},
"fields" : { "#datacenter" : "amazon-east1" }
}
EOF
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec ^
-H "Authorization: Bearer $INGEST_TOKEN" ^
-H "Content-Type: text/plain; charset=utf-8" ^
-d @'{ ^
"time" : 1537537729.0, ^
"event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver", ^
"source" : "/var/log/application.log", ^
"sourcetype" : "applog", ^
"fields" : { "#env" : "prod" } ^
} ^
^
{ ^
"time" : 1537535729.0, ^
"event" : { ^
"message" : "System shutdown", ^
"host" : { "ip" : "127.0.0.1", "port" : 2222 } ^
}, ^
"fields" : { "#datacenter" : "amazon-east1" } ^
} '
curl.exe -X POST
-H "Authorization: Bearer $INGEST_TOKEN"
-H "Content-Type: text/plain; charset=utf-8"
-d '{
"time" : 1537537729.0,
"event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
"source" : "/var/log/application.log",
"sourcetype" : "applog",
"fields" : { "#env" : "prod" }
}
{
"time" : 1537535729.0,
"event" : {
"message" : "System shutdown",
"host" : { "ip" : "127.0.0.1", "port" : 2222 }
},
"fields" : { "#datacenter" : "amazon-east1" }
}'
"$YOUR_LOGSCALE_URL/api/v1/ingest/hec"
#!/usr/bin/perl
use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec';
my $json = '{
"time" : 1537537729.0,
"event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
"source" : "/var/log/application.log",
"sourcetype" : "applog",
"fields" : { "#env" : "prod" }
}
{
"time" : 1537535729.0,
"event" : {
"message" : "System shutdown",
"host" : { "ip" : "127.0.0.1", "port" : 2222 }
},
"fields" : { "#datacenter" : "amazon-east1" }
}';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Authorization" => "Bearer $INGEST_TOKEN");
$req->header("Content-Type" => "text/plain; charset=utf-8");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";
#! /usr/local/bin/python3
import requests
url = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec'
mydata = r'''{
"time" : 1537537729.0,
"event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
"source" : "/var/log/application.log",
"sourcetype" : "applog",
"fields" : { "#env" : "prod" }
}
{
"time" : 1537535729.0,
"event" : {
"message" : "System shutdown",
"host" : { "ip" : "127.0.0.1", "port" : 2222 }
},
"fields" : { "#datacenter" : "amazon-east1" }
}'''
resp = requests.post(url,
data = mydata,
headers = {
"Authorization" : "Bearer $INGEST_TOKEN",
"Content-Type" : "text/plain; charset=utf-8"
}
)
print(resp.text)
const https = require('https');
const data = JSON.stringify(
{
"time" : 1537537729.0,
"event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
"source" : "/var/log/application.log",
"sourcetype" : "applog",
"fields" : { "#env" : "prod" }
}
{
"time" : 1537535729.0,
"event" : {
"message" : "System shutdown",
"host" : { "ip" : "127.0.0.1", "port" : 2222 }
},
"fields" : { "#datacenter" : "amazon-east1" }
}
);
const options = {
hostname: '$YOUR_LOGSCALE_URL/api/v1/ingest/hec',
path: '/graphql',
port: 443,
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': data.length,
Authorization: 'BEARER ' + process.env.TOKEN,
'User-Agent': 'Node',
},
};
const req = https.request(options, (res) => {
let data = '';
console.log(`statusCode: ${res.statusCode}`);
res.on('data', (d) => {
data += d;
});
res.on('end', () => {
console.log(JSON.parse(data).data);
});
});
req.on('error', (error) => {
console.error(error);
});
req.write(data);
req.end();
Raw HEC
LogScale's Raw HEC API is a simple line-delimited ingest endpoint for unstructured logs.
You will find it at /api/v1/ingest/hec/raw
and at
/services/collector/raw
.
Simply send a POST to one of the two endpoints above. Each line in the
input (separated by /n
,
/r
, or
/r/n
) will be ingested as an
event.
You can optionally add an
X-Splunk-Request-Channel
header or
channel
as a query parameter.
These will be added as a field on the event named "channel".
Example
In a simple text editor, create a file named,
events.txt
and copy the
following lines into it:
Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver
System shutdown
Then execute the following from the command-line:
curl $YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo \
-X POST \
-H "Content-Type: text/plain; charset=utf-8" \
-H "Authorization: Bearer $INGEST_TOKEN" \
--data-binary "@events.txt"