HTTP Event Collector (HEC)

LogScale's HEC API is an alternative HTTP ingest endpoint. You will find it at /api/v1/ingest/hec and at /services/collector.

Note

The best practices described in Ingest via API Best Practices also apply to the HEC API.

Format of Data

Ingested data is a series of whitespace delimited JSON objects, containing one or more of the following elements. All elements are optional.

Member	Description
`time`	Time in seconds since January 1, 1970 in UTC. This can be a integer or floating point number to support milliseconds. LogScale represents time with millisecond precision. Times are interpreted as follows: `Integer` — treated as seconds `Float/Double` — treated as seconds plus milliseconds `Integer` — parsed to an integer or float and then treated as above
`timezone`	Can be used to describe the time zone in which the event happened. Defaults to `Z` (i.e., UTC).
`index`	Optional name of the repository to ingest into. In public-facing API's this must — if present — be equal to the repository used to create the ingest token used for authentication. In private cluster setups, humio can be configured to allow these to be different. See below.
`sourcetype`	Translated to #type inside LogScale. If set, this is used to choose which LogScale parser to use for extracting fields. To refer to a parser within the given repository just the name can be provided. To refer to the parser from a package installed into the repository, use the form `PACKAGESCOPE/PACKAGENAME:PARSERNAME`. For example, to use the `apache_access` within the `apache/http-server` package, use apache/http-server:apache_access.
`source`	Translated to the @source field in LogScale. Typically used to designate the path to the file that is being shipped to LogScale.
`host`	Translated to the @host field in LogScale. Typically used to designate the origin host.
`event`	This can JSON Object, a String, array or object. Translated to the @rawstring field in LogScale. When this is a JSON Object, all members of the object will become accessible fields in humio with no further processing. If it is a string, the key/value parser is always applied to the string to extract elements. The key/value parser searches for `key=value`, `key="value"` or `key='value'`.
`fields`	JSON object containing extra fields to the event. This can be used if `event` is a string, boolean or number and it is pre-processed prior to ingest to extract fields. Tags #tags can be added to the event by specifying fields starting with `#`. It cannot be used if `event` is an array or an object.

Data Handling for Large Data Events

The following examples detail the resulting ingested data for different raw event volumes. For more information on limits, see Limits & Standards.

Ingest Data within Limits

Mac OS or Linux (curl)

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain; charset=utf-8" \
    -d @- << EOF
{
    "fields": {
        "#someTag": "a-small-value",
        "someField1": "tiny-data-value",
        "someField2": "small-data-value"
    },
    "event": "also-a-tiny-data-value"
}
EOF

Mac OS or Linux (curl) One-line

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain; charset=utf-8" \
    -d @- << EOF
{
    "fields": {
        "#someTag": "a-small-value",
        "someField1": "tiny-data-value",
        "someField2": "small-data-value"
    },
    "event": "also-a-tiny-data-value"
}
EOF

Windows Cmd and curl

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec ^
    -H "Authorization: Bearer $INGEST_TOKEN" ^
    -H "Content-Type: text/plain; charset=utf-8" ^
    -d @'{ ^
    "fields": { ^
        "#someTag": "a-small-value", ^
        "someField1": "tiny-data-value", ^
        "someField2": "small-data-value" ^
    }, ^
    "event": "also-a-tiny-data-value" ^
} '

Windows Powershell and curl

powershell

curl.exe -X POST 
    -H "Authorization: Bearer $INGEST_TOKEN"
    -H "Content-Type: text/plain; charset=utf-8"
    -d '{
    "fields": {
        "#someTag": "a-small-value",
        "someField1": "tiny-data-value",
        "someField2": "small-data-value"
    },
    "event": "also-a-tiny-data-value"
}'
"$YOUR_LOGSCALE_URL/api/v1/ingest/hec"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec';
my $json = '{
    "fields": {
        "#someTag": "a-small-value",
        "someField1": "tiny-data-value",
        "someField2": "small-data-value"
    },
    "event": "also-a-tiny-data-value"
}';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Authorization" => "Bearer $INGEST_TOKEN");
$req->header("Content-Type" => "text/plain; charset=utf-8");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec'
mydata = r'''{
    "fields": {
        "#someTag": "a-small-value",
        "someField1": "tiny-data-value",
        "someField2": "small-data-value"
    },
    "event": "also-a-tiny-data-value"
}'''

resp = requests.post(url,
                     data = mydata,
                     headers = {
   "Authorization" : "Bearer $INGEST_TOKEN",
   "Content-Type" : "text/plain; charset=utf-8"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');

const data = JSON.stringify(
    {
    "fields": {
        "#someTag": "a-small-value",
        "someField1": "tiny-data-value",
        "someField2": "small-data-value"
    },
    "event": "also-a-tiny-data-value"
}
);


const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/ingest/hec',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();

Example Responses

Success (HTTP Response Code 200 OK)

Field	Value
#repo	developer
#type	none
@id	L4tQKldzrldCwMCVc8PIq7Wp_0_0_1719304163
@ingesttimestamp	1719304163717
@rawstring	also-a-tiny-data-value
@timestamp	1719304163717
@timestamp.nanos	0
@timezone	Z
#someTag	a-small-value
someField1	tiny-data-value
someField2	small-data-value

All expected data and fields are present, and they are not truncated.

Raw String is Too Large

Mac OS or Linux (curl)

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain; charset=utf-8" \
    -d @- << EOF
{
    "fields": {
	"#someTag": "a-small-value",
        "someField1": "someValue1",
        "someField2": "someValue2"
    },
    "event": "... More than 1MB of data ..."
}
EOF

Mac OS or Linux (curl) One-line

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain; charset=utf-8" \
    -d @- << EOF
{
    "fields": {
	"#someTag": "a-small-value",
        "someField1": "someValue1",
        "someField2": "someValue2"
    },
    "event": "... More than 1MB of data ..."
}
EOF

Windows Cmd and curl

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec ^
    -H "Authorization: Bearer $INGEST_TOKEN" ^
    -H "Content-Type: text/plain; charset=utf-8" ^
    -d @'{ ^
    "fields": { ^
	"#someTag": "a-small-value", ^
        "someField1": "someValue1", ^
        "someField2": "someValue2" ^
    }, ^
    "event": "... More than 1MB of data ..." ^
} '

Windows Powershell and curl

powershell

curl.exe -X POST 
    -H "Authorization: Bearer $INGEST_TOKEN"
    -H "Content-Type: text/plain; charset=utf-8"
    -d '{
    "fields": {
	"#someTag": "a-small-value",
        "someField1": "someValue1",
        "someField2": "someValue2"
    },
    "event": "... More than 1MB of data ..."
}'
"$YOUR_LOGSCALE_URL/api/v1/ingest/hec"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec';
my $json = '{
    "fields": {
	"#someTag": "a-small-value",
        "someField1": "someValue1",
        "someField2": "someValue2"
    },
    "event": "... More than 1MB of data ..."
}';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Authorization" => "Bearer $INGEST_TOKEN");
$req->header("Content-Type" => "text/plain; charset=utf-8");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec'
mydata = r'''{
    "fields": {
	"#someTag": "a-small-value",
        "someField1": "someValue1",
        "someField2": "someValue2"
    },
    "event": "... More than 1MB of data ..."
}'''

resp = requests.post(url,
                     data = mydata,
                     headers = {
   "Authorization" : "Bearer $INGEST_TOKEN",
   "Content-Type" : "text/plain; charset=utf-8"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');

const data = JSON.stringify(
    {
    "fields": {
	"#someTag": "a-small-value",
        "someField1": "someValue1",
        "someField2": "someValue2"
    },
    "event": "... More than 1MB of data ..."
}
);


const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/ingest/hec',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();

Example Responses

Success (HTTP Response Code 200 OK)

Field	Value
#error	true
#repo	developer
#type	json
@error	true
@error_msg	Input too big. Event truncated at 1048576
@error_msg[0]	Input too big. Event truncated at 1048576
@id	GveuGB8hexKWpraoahisoA46_0_0_1719301041
@ingesttimestamp	1719301041888
@input_size	1048677
@rawstring	Truncated data
@timestamp	1719301041888
@timestamp.nanos	0
@timezone	Z
#someTag	a-small-value

The someField1 and someField2 are not present, but #someTag is.

Field is Too Large

Mac OS or Linux (curl)

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain; charset=utf-8" \
    -d @- << EOF
{
    "fields": {
	"#someTag": "a-small-value",
        "someField1": "tiny-data-value",
        "someField2": "... More than 1MB of data ..."
    },
    "event": "also-a-tiny-data-value"
}
EOF

Mac OS or Linux (curl) One-line

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain; charset=utf-8" \
    -d @- << EOF
{
    "fields": {
	"#someTag": "a-small-value",
        "someField1": "tiny-data-value",
        "someField2": "... More than 1MB of data ..."
    },
    "event": "also-a-tiny-data-value"
}
EOF

Windows Cmd and curl

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec ^
    -H "Authorization: Bearer $INGEST_TOKEN" ^
    -H "Content-Type: text/plain; charset=utf-8" ^
    -d @'{ ^
    "fields": { ^
	"#someTag": "a-small-value", ^
        "someField1": "tiny-data-value", ^
        "someField2": "... More than 1MB of data ..." ^
    }, ^
    "event": "also-a-tiny-data-value" ^
} '

Windows Powershell and curl

powershell

curl.exe -X POST 
    -H "Authorization: Bearer $INGEST_TOKEN"
    -H "Content-Type: text/plain; charset=utf-8"
    -d '{
    "fields": {
	"#someTag": "a-small-value",
        "someField1": "tiny-data-value",
        "someField2": "... More than 1MB of data ..."
    },
    "event": "also-a-tiny-data-value"
}'
"$YOUR_LOGSCALE_URL/api/v1/ingest/hec"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec';
my $json = '{
    "fields": {
	"#someTag": "a-small-value",
        "someField1": "tiny-data-value",
        "someField2": "... More than 1MB of data ..."
    },
    "event": "also-a-tiny-data-value"
}';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Authorization" => "Bearer $INGEST_TOKEN");
$req->header("Content-Type" => "text/plain; charset=utf-8");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec'
mydata = r'''{
    "fields": {
	"#someTag": "a-small-value",
        "someField1": "tiny-data-value",
        "someField2": "... More than 1MB of data ..."
    },
    "event": "also-a-tiny-data-value"
}'''

resp = requests.post(url,
                     data = mydata,
                     headers = {
   "Authorization" : "Bearer $INGEST_TOKEN",
   "Content-Type" : "text/plain; charset=utf-8"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');

const data = JSON.stringify(
    {
    "fields": {
	"#someTag": "a-small-value",
        "someField1": "tiny-data-value",
        "someField2": "... More than 1MB of data ..."
    },
    "event": "also-a-tiny-data-value"
}
);


const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/ingest/hec',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();

Example Responses

Success (HTTP Response Code 200 OK)

Field	Value
#error	true
#repo	developer
#type	json
@error	true
@error_msg	Input too big. Event truncated at 1048576
@error_msg[0]	Input too big. Event truncated at 1048576
@id	GveuGB8hexKWpraoahisoA46_1_1_1719303197
@ingesttimestamp	1719303197844
@input_size	1049579
@rawstring	also-a-tiny-data-value
@timestamp	1719303197844
@timestamp.nanos	0
@timezone	Z
#someTag	a-small-value

The someField1 and someField2 are not present, but #someTag is.

Tag is Too Large

Mac OS or Linux (curl)

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain; charset=utf-8" \
    -d @- << EOF
{
    "fields": {
	"#someTag": "... More than 65535 bytes of data ...",
      "someField1": "tiny-data-value",
      "someField2": "small-data-value"
    },
    "event": "also-a-tiny-data-value"
}
EOF

Mac OS or Linux (curl) One-line

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain; charset=utf-8" \
    -d @- << EOF
{
    "fields": {
	"#someTag": "... More than 65535 bytes of data ...",
      "someField1": "tiny-data-value",
      "someField2": "small-data-value"
    },
    "event": "also-a-tiny-data-value"
}
EOF

Windows Cmd and curl

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec ^
    -H "Authorization: Bearer $INGEST_TOKEN" ^
    -H "Content-Type: text/plain; charset=utf-8" ^
    -d @'{ ^
    "fields": { ^
	"#someTag": "... More than 65535 bytes of data ...", ^
      "someField1": "tiny-data-value", ^
      "someField2": "small-data-value" ^
    }, ^
    "event": "also-a-tiny-data-value" ^
} '

Windows Powershell and curl

powershell

curl.exe -X POST 
    -H "Authorization: Bearer $INGEST_TOKEN"
    -H "Content-Type: text/plain; charset=utf-8"
    -d '{
    "fields": {
	"#someTag": "... More than 65535 bytes of data ...",
      "someField1": "tiny-data-value",
      "someField2": "small-data-value"
    },
    "event": "also-a-tiny-data-value"
}'
"$YOUR_LOGSCALE_URL/api/v1/ingest/hec"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec';
my $json = '{
    "fields": {
	"#someTag": "... More than 65535 bytes of data ...",
      "someField1": "tiny-data-value",
      "someField2": "small-data-value"
    },
    "event": "also-a-tiny-data-value"
}';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Authorization" => "Bearer $INGEST_TOKEN");
$req->header("Content-Type" => "text/plain; charset=utf-8");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec'
mydata = r'''{
    "fields": {
	"#someTag": "... More than 65535 bytes of data ...",
      "someField1": "tiny-data-value",
      "someField2": "small-data-value"
    },
    "event": "also-a-tiny-data-value"
}'''

resp = requests.post(url,
                     data = mydata,
                     headers = {
   "Authorization" : "Bearer $INGEST_TOKEN",
   "Content-Type" : "text/plain; charset=utf-8"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');

const data = JSON.stringify(
    {
    "fields": {
	"#someTag": "... More than 65535 bytes of data ...",
      "someField1": "tiny-data-value",
      "someField2": "small-data-value"
    },
    "event": "also-a-tiny-data-value"
}
);


const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/ingest/hec',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();

Example Responses

Failure (HTTP Response Code 500 Error; Can be retried)

none

Event data is too large

Entire request is dropped, no data in LogScale.

Authentication

You will need to provide a Ingest Tokens in the HTTP Authorization header.

The ingest token contains the name of the repository the data is stored in, and ingested events will be stored in the repository corresponding to the ingest token.

If using an Organization API Token with the Ingest across all repositories within organization permission, then HEC allows ingest to any repository specified as "index": "repository-name" in the body of a message, as long as the ingest token is valid for any repository on the LogScale cluster. If the named repository does not exist then an error will be returned.

This is a potential security issue on a public API endpoint, so this option should only be used inside a trusted environment.

Example

Below is an example of the sending a JSON document for ingest:

Mac OS or Linux (curl)

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain; charset=utf-8" \
    -d @- << EOF
{
  "time" : 1537537729.0,
  "event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
  "source" : "/var/log/application.log",
  "sourcetype" : "applog",
  "fields" : { "#env" : "prod" }
}

{
  "time" : 1537535729.0,
  "event" : {
    "message" : "System shutdown",
    "host" : { "ip" : "127.0.0.1", "port" : 2222 }
  },
  "fields" : { "#datacenter" : "amazon-east1" }
}
EOF

Mac OS or Linux (curl) One-line

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain; charset=utf-8" \
    -d @- << EOF
{
  "time" : 1537537729.0,
  "event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
  "source" : "/var/log/application.log",
  "sourcetype" : "applog",
  "fields" : { "#env" : "prod" }
}

{
  "time" : 1537535729.0,
  "event" : {
    "message" : "System shutdown",
    "host" : { "ip" : "127.0.0.1", "port" : 2222 }
  },
  "fields" : { "#datacenter" : "amazon-east1" }
}
EOF

Windows Cmd and curl

shell

curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec ^
    -H "Authorization: Bearer $INGEST_TOKEN" ^
    -H "Content-Type: text/plain; charset=utf-8" ^
    -d @'{ ^
  "time" : 1537537729.0, ^
  "event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver", ^
  "source" : "/var/log/application.log", ^
  "sourcetype" : "applog", ^
  "fields" : { "#env" : "prod" } ^
} ^
 ^
{ ^
  "time" : 1537535729.0, ^
  "event" : { ^
    "message" : "System shutdown", ^
    "host" : { "ip" : "127.0.0.1", "port" : 2222 } ^
  }, ^
  "fields" : { "#datacenter" : "amazon-east1" } ^
} '

Windows Powershell and curl

powershell

curl.exe -X POST 
    -H "Authorization: Bearer $INGEST_TOKEN"
    -H "Content-Type: text/plain; charset=utf-8"
    -d '{
  "time" : 1537537729.0,
  "event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
  "source" : "/var/log/application.log",
  "sourcetype" : "applog",
  "fields" : { "#env" : "prod" }
}

{
  "time" : 1537535729.0,
  "event" : {
    "message" : "System shutdown",
    "host" : { "ip" : "127.0.0.1", "port" : 2222 }
  },
  "fields" : { "#datacenter" : "amazon-east1" }
}'
"$YOUR_LOGSCALE_URL/api/v1/ingest/hec"

Perl

perl

#!/usr/bin/perl

use HTTP::Request;
use LWP;
my $TOKEN = "TOKEN";
my $uri = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec';
my $json = '{
  "time" : 1537537729.0,
  "event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
  "source" : "/var/log/application.log",
  "sourcetype" : "applog",
  "fields" : { "#env" : "prod" }
}

{
  "time" : 1537535729.0,
  "event" : {
    "message" : "System shutdown",
    "host" : { "ip" : "127.0.0.1", "port" : 2222 }
  },
  "fields" : { "#datacenter" : "amazon-east1" }
}';
my $req = HTTP::Request->new("POST", $uri );
$req->header("Authorization" => "Bearer $INGEST_TOKEN");
$req->header("Content-Type" => "text/plain; charset=utf-8");
$req->content( $json );
my $lwp = LWP::UserAgent->new;
my $result = $lwp->request( $req );
print $result->{"_content"},"\n";

Python

python

#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec'
mydata = r'''{
  "time" : 1537537729.0,
  "event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
  "source" : "/var/log/application.log",
  "sourcetype" : "applog",
  "fields" : { "#env" : "prod" }
}

{
  "time" : 1537535729.0,
  "event" : {
    "message" : "System shutdown",
    "host" : { "ip" : "127.0.0.1", "port" : 2222 }
  },
  "fields" : { "#datacenter" : "amazon-east1" }
}'''

resp = requests.post(url,
                     data = mydata,
                     headers = {
   "Authorization" : "Bearer $INGEST_TOKEN",
   "Content-Type" : "text/plain; charset=utf-8"
}
)

print(resp.text)

Node.js

javascript

const https = require('https');

const data = JSON.stringify(
    {
  "time" : 1537537729.0,
  "event" : "Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver",
  "source" : "/var/log/application.log",
  "sourcetype" : "applog",
  "fields" : { "#env" : "prod" }
}

{
  "time" : 1537535729.0,
  "event" : {
    "message" : "System shutdown",
    "host" : { "ip" : "127.0.0.1", "port" : 2222 }
  },
  "fields" : { "#datacenter" : "amazon-east1" }
}
);


const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/ingest/hec',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();

Raw HEC

LogScale's Raw HEC API is a simple line-delimited ingest endpoint for unstructured logs.

You will find it at /api/v1/ingest/hec/raw and at /services/collector/raw.

Simply send a POST to one of the two endpoints above. Each line in the input (separated by /n, /r, or /r/n) will be ingested as an event.

You can optionally add an X-Splunk-Request-Channel header or channel as a query parameter. These will be added as a field on the event named "channel".

Example

In a simple text editor, create a file named, events.txt and copy the following lines into it:

ini

Fri, 21 Sep 2018 13:48:49 GMT - system started name=webserver
System shutdown

Then execute the following from the command-line:

logscale

curl $YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo \
  -X POST \
  -H "Content-Type: text/plain; charset=utf-8" \
  -H "Authorization: Bearer $INGEST_TOKEN" \
  --data-binary "@events.txt"

LogScale APIs 1.118.0-1.147.0

APIs

API Authentication

Cluster Management API

Health Check API

Ingest API

Lookup API

Redact Events API

Search API

Software Libraries