Creating a new ingest listener is all about mapping a port on a network interface through a parser to a repository.

Figure 86. Create Listener

To reduce packet loss in bursts of UDP traffic, please increase the maximum allowed receive buffer size for UDP.

LogScale will try to increase the buffer to up to 128MB, but will accept whatever the system sets as maximum.

# Get the current limit from the kernel (in bytes)
$ sysctl net.core.rmem_max
# Set to 16MB. Decide on a value of (say) 0.5 - 2 seconds worth of inbound UDP packets.
$ sudo sysctl net.core.rmem_max=16777216

net.core.rmem_max=16777216

You can ingest events using one of the many Package Marketplace but when your requirements do not match, you can supply a stream of events on TCP, separated by line feeds. This API allows you to create and configure a TCP listener for such events.

Use cases include accepting rsyslogd forward format and similar plain-text event streams.

GET    /api/v1/listeners
POST   /api/v1/listeners
GET    /api/v1/listeners/$ID
DELETE /api/v1/listeners/$ID

If you use rsyslog for the transport of logs, this example serves as a starting point:

Using a simple text editor, create a file named, create-rsyslogd-rfc3339-parser.json and copy the following lines into it:

{
  "parser": "^(?pri\\\\d+)(?datetimestring\\\\S+) (?host\\\\S*) (?syslogtag\\\\S*): ?(?message.*)",
  "kind": "regex",
  "parseKeyValues": true,
  "dateTimeFormat": "yyyy-MM-dd'T'HH:mm:ss[.SSSSSS]XXX",
  "dateTimeFields": ["datetimestring"]
}

Then execute the following from the command-line:

curl -XPOST \
 -d @create-rsyslogd-rfc3339-parser.json \
 -H "Authorization: Bearer $TOKEN" \
 -H 'Content-Type: application/json' \
 "$YOUR_LOGSCALE_URL/api/v1/repositories/$REPOSITORY_NAME/parsers/rsyslogd-rfc3339"

Example setting up a listener using the rsyslogd forward format added above.

Using a simple text editor, create a file named, create-rsyslogd-listener.json and copy the following lines into it:

{
  "listenerPort": 7777,
  "kind": "tcp",
  "dataspaceID": "$REPOSITORY_NAME",
  "parser": "rsyslogd-rfc3339",
  "bindInterface": "0.0.0.0",
  "name": "my rsyslog input",
  "vhost": 1
}

The setting bindInterface is optional. If set, sets local interface to bind on to select network interface. Also, vhost is optional. If set, only the cluster node with that index binds the port.

Execute the following from the command-line:

curl -XPOST \
 -d @create-rsyslogd-listener.json \
 -H 'Content-Type: application/json' \
 -H "Authorization: Bearer $TOKEN" \
 '$YOUR_LOGSCALE_URL/api/v1/listeners'

curl -H "Authorization: Bearer $TOKEN" "$YOUR_LOGSCALE_URL/api/v1/listeners"
curl -H "Authorization: Bearer $TOKEN" "$YOUR_LOGSCALE_URL/api/v1/listeners/tcp7777"

Listeners also support UDP by setting kind to udp. For UDP, each UDP datagram is ingested as a single log line (it is not split by newlines).

It is possible to specify that fields in the incoming events should be turned into tags. This can be done by setting "tagFields": ["fielda", "fieldb"] when creating a listener. Only use tags like this if you really need them.

To reduce packet loss in bursts of UDP traffic, increase the maximum allowed receive buffer size for UDP. LogScale will try to increase the buffer up to 128 MB but will accept whatever the system sets as maximum.

# To set to 16 MB.
sudo sysctl net.core.rmem_max=16777216