Ingesting with Raw HEC

LogScale's Raw HEC API is a simple line-delimited ingest endpoint for unstructured logs.

You will find it at /api/v1/ingest/hec/raw and at /services/collector/raw.

Simply send a POST to one of the two endpoints above. Each line in the input (separated by /n, /r, or /r/n) will be ingested as an event.

You can optionally add an X-Splunk-Request-Channel header or channel as a query parameter. These will be added as a field on the event named "channel".

Example

Note that you can send data without wrapping it in JSON, for example:

Mac OS or Linux (curl)
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain" \
    -d '2024-10-14 12:01:21 INFO: Application started.'
Mac OS or Linux (curl) One-line
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain" \
    -d '2024-10-14 12:01:21 INFO: Application started.'
Windows Cmd and curl
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo ^
    -H "Authorization: Bearer $INGEST_TOKEN" ^
    -H "Content-Type: text/plain" ^
    -d '2024-10-14 12:01:21 INFO: Application started.'
Windows Powershell and curl
powershell
curl.exe -X POST 
    -H "Authorization: Bearer $INGEST_TOKEN"
    -H "Content-Type: text/plain"
    -d '2024-10-14 12:01:21 INFO: Application started.'
    "$YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo"
Perl
perl
#!/usr/bin/perl

use HTTP::Request;
use LWP;

my $INGEST_TOKEN = "TOKEN";

my $uri = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo';

my $json = '2024-10-14 12:01:21 INFO: Application started.';
my $req = HTTP::Request->new("POST", $uri );

$req->header("Authorization" => "Bearer $INGEST_TOKEN");
$req->header("Content-Type" => "text/plain");

$req->content( $json );

my $lwp = LWP::UserAgent->new;

my $result = $lwp->request( $req );

print $result->{"_content"},"\n";
Python
python
#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo'
mydata = r'''2024-10-14 12:01:21 INFO: Application started.'''

resp = requests.post(url,
                     data = mydata,
                     headers = {
   "Authorization" : "Bearer $INGEST_TOKEN",
   "Content-Type" : "text/plain"
}
)

print(resp.text)
Node.js
javascript
const https = require('https');

const data = JSON.stringify(
    2024-10-14 12:01:21 INFO: Application started.
);


const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();

Multiple log lines are also supported, as shown in the following example:

Mac OS or Linux (curl)
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain" \
    -d @- << EOF
2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: Helper Tool Initialized.
2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: New Connection requested from pid: 33184
EOF
Mac OS or Linux (curl) One-line
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo \
    -H "Authorization: Bearer $INGEST_TOKEN" \
    -H "Content-Type: text/plain" \
    -d @- << EOF
2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: Helper Tool Initialized.
2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: New Connection requested from pid: 33184
EOF
Windows Cmd and curl
shell
curl -v -X POST $YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo ^
    -H "Authorization: Bearer $INGEST_TOKEN" ^
    -H "Content-Type: text/plain" ^
    -d @'2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: Helper Tool Initialized. ^
2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: New Connection requested from pid: 33184 '
Windows Powershell and curl
powershell
curl.exe -X POST 
    -H "Authorization: Bearer $INGEST_TOKEN"
    -H "Content-Type: text/plain"
    -d '2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: Helper Tool Initialized.
2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: New Connection requested from pid: 33184'
    "$YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo"
Perl
perl
#!/usr/bin/perl

use HTTP::Request;
use LWP;

my $INGEST_TOKEN = "TOKEN";

my $uri = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo';

my $json = '2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: Helper Tool Initialized.
2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: New Connection requested from pid: 33184';
my $req = HTTP::Request->new("POST", $uri );

$req->header("Authorization" => "Bearer $INGEST_TOKEN");
$req->header("Content-Type" => "text/plain");

$req->content( $json );

my $lwp = LWP::UserAgent->new;

my $result = $lwp->request( $req );

print $result->{"_content"},"\n";
Python
python
#! /usr/local/bin/python3

import requests

url = '$YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo'
mydata = r'''2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: Helper Tool Initialized.
2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: New Connection requested from pid: 33184'''

resp = requests.post(url,
                     data = mydata,
                     headers = {
   "Authorization" : "Bearer $INGEST_TOKEN",
   "Content-Type" : "text/plain"
}
)

print(resp.text)
Node.js
javascript
const https = require('https');

const data = JSON.stringify(
    2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: Helper Tool Initialized.
2024-10-18 06:51:33 [Helper Tool] Info CD063E71-9F0A-4861-99A8-28204DE1234: New Connection requested from pid: 33184
);


const options = {
  hostname: '$YOUR_LOGSCALE_URL/api/v1/ingest/hec/raw?channel=foo',
  path: '/graphql',
  port: 443,
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length,
    Authorization: 'BEARER ' + process.env.TOKEN,
    'User-Agent': 'Node',
  },
};

const req = https.request(options, (res) => {
  let data = '';
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', (d) => {
    data += d;
  });
  res.on('end', () => {
    console.log(JSON.parse(data).data);
  });
});

req.on('error', (error) => {
  console.error(error);
});

req.write(data);
req.end();

This creates two unique events in LogScale.