Sinks (sinks
)
The sinks
block configures the
sinks (where the data will be sent) that are used by the source or
sources.
Note
When syslog is the data source, only one sink can be configured per syslog data source.
# Define the sink (destination) for the logs
sinks:
example_sink:
type: logscale
url: "https://cloud.humio.com/" # Replace with your LogScale instance URL
token: "${INGEST_TOKEN}" # Use environment variable for the ingest token
# Optional: Override the maximum batch size (in bytes)
# maxBatchSize: 8388608 # 8 MB
# Default is 16 MB. It should not be increased, but may be lowered if needed.
# Optional: Override the maximum event size (in bytes)
# maxEventSize: 1048576 # 1 MB
# Default is 1 MB, which is the default maximum that LogScale supports.
# Optional: Specify compression method
# compression: "gzip" # Options: "auto", "none", "gzip", "deflate"
# Default is "auto", which attempts "gzip" but falls back to "none" if unsupported.
# Optional: Specify compression level (0-9, where 0 is no compression and 9 is best compression)
# compressionLevel: 1
# Optional: Specify a proxy
# proxy: "http://proxy.example.com:8080"
# Default is "auto", which uses the system proxy if possible, or falls back to "none".
# You can also use "system" to force system proxy use, or "none" to disable proxy use.
# Optional: Configure TLS options
# tls:
# insecure: false # Set to true to disable certificate validation (not recommended)
# # Note: caCert and caFile are mutually exclusive. Use only one of them.
# caCert: "-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----"
# # caCert is for providing an inline PEM encoded CA certificate
# # caFile: "/path/to/ca/cert.pem"
# # caFile is for providing a path to a file containing a PEM encoded CA certificate
# Optional: Number of worker threads for sending data
# workers: 4
# Configure the queue for buffering events
queue:
# Memory queue configuration
type: memory
maxLimitInMB: 1024 # Maximum queue size in MB
# Note: The queue size can be lowered if needed, but it should not be necessary to increase it.
# Optional: Action to take when queue is full
# fullAction: "pause" # Options: "pause" or "deleteOldest"
# Optional: Flush timeout in milliseconds
# flushTimeOutInMillisecond: 1000
# Alternate disk queue configuration (uncomment to use)
# queue:
# type: disk
# maxLimitInMB: 4096 # Maximum queue size in MB
# # Note: Changing the disk queue size requires a rewrite of the queue storage file.
# # Optional: Action to take when queue is full
# # fullAction: "pause" # Options: "pause" or "deleteOldest"
# # Optional: Storage directory
# # storageDir: "/path/to/queue/storage"
# # Default storage directory is under the dataDirectory of the program.
my_examplesink1:
type: logscale
url: "https://cloud.humio.com/" # Replace with your LogScale instance URL
token: "${INGEST_TOKEN}" # Use environment variable for the ingest token
# Define the sources for data collection
sources:
# Add your sources here. Examples include:
# - file
# - syslog
# - wineventlog
# - journald
# - cmd
# - unifiedlog
The MySinksName
is a top level element which
contains each of your sink configurations.
Table: Sinks
The elements listed in this table define how each sink is configured.
Table: MySinkName
Parameter | Type | Required | Default Value | Description |
---|---|---|---|---|
compression | string | optional[a] | auto | Specify the type of data compression, possible values: auto, none, gzip, deflate. |
compressionLevel | integer | optional[a] | false | Specify the level of compression where 1 is best speed and 9 is best compression, if set to undefined or 0 the default value for the compression algorithm, specified in compression, is applied. |
maxBatchSize | integer | optional[a] | 16MB | Specifies the max size of batch (default 16MB) and works along with the maximum events per request. The limits are also propagated to all the sources that reference the sink. |
maxEventSize | integer | optional[a] | 1MB | This sets the maximum allowed single event size to 1 MB; larger messages will be truncated. Syslog and syslog TLS logs sources only, if maxEventSize is also defined at source level the lower of the two values will be applied. |
proxy | string | optional[a] | system | Must be set to none for Windows Server.Otherwise this can be used to specify an override proxy configuration for the sink, possible values: system , none or a URL such as: http://127.0.0.1:3129 for a http proxy. The default is system, which will try to determine the appropriate proxy or fallback to none. |
tls | string | optional[a] | This object contains details on the PEM certificates. this section allows you to override the defaults. Only one of the following options should be specified: | |
Valid Values | ||||
caCert | Specify this key to load a certificate from the config file. | |||
caFile | Specify this key to load the PEM certificate from an external file. | |||
insecure | Specify if certificate validation is needed, if set to true the certificate validation is skipped. | |||
token | string | optional[a] | Specify the Ingest Tokens for your repository or an environment variable. | |
type | string | optional[a] | Specify the type of sink. This can be set to humio or logscale . | |
Valid Values | ||||
humio | ||||
url | string | optional[a] | Specify the url of your LogScale account for example https://cloud.humio.com. | |
workers | string | optional[a] | 4 | Specifies how many workers to use to send to LogScale, under normal circumstances leave this at the default setting, for more information see Sink Workers |
[a] Optional parameters use their default value unless explicitly set. |
Using a Proxy Server
The Falcon LogScale Collector supports using a forward proxy server when sending logs using the sink. In some environments, where direct access to LogScale is prohibited, it may be necessary to configure the proxy server manually. The collector attempts to detect the system's proxy automatically. If the collector should use a different proxy than the system's, or instead connect directly, it must be specified in the sink configuration. The proxy option accepts the following keywords: auto, system, and none, but it also accepts a URL specifying the proxy server to use.