Sinks (sinks)

The sinks block configures the sinks (where the data will be sent) that are used by the source or sources.

Note

When syslog is the data source, only one sink can be configured per syslog data source.

The LogScale ingest APIs currently transport data over HTTP to the same ports that are used for the web interface for LogScale, no special ports need to be configured. By default the data is compressed and requires HTTPS, although these can be configured.

yaml
# Define the sink (destination) for the logs
sinks:
  example_sink:
    type: logscale 
    url: "https://cloud.humio.com/"  # Replace with your LogScale instance URL
    token: "${INGEST_TOKEN}"  # Use environment variable for the ingest token

    # Optional: Override the maximum batch size (in bytes)
    # maxBatchSize: 8388608  # 8 MB
    # Default is 16 MB. It should not be increased, but may be lowered if needed.

    # Optional: Override the maximum event size (in bytes)
    # maxEventSize: 1048576  # 1 MB
    # Default is 1 MB, which is the default maximum that LogScale supports.

    # Optional: Specify compression method
    # compression: "gzip"  # Options: "auto", "none", "gzip", "deflate"
    # Default is "auto", which attempts "gzip" but falls back to "none" if unsupported.

    # Optional: Specify compression level (0-9, where 0 is no compression and 9 is best compression)
    # compressionLevel: 1

    # Optional: Specify a proxy
    # proxy: "http://proxy.example.com:8080"
    # Default is "auto", which uses the system proxy if possible, or falls back to "none".
    # You can also use "system" to force system proxy use, or "none" to disable proxy use.

    # Optional: Configure TLS options
    # tls:
    #   insecure: false  # Set to true to disable certificate validation (not recommended)
    #   # Note: caCert and caFile are mutually exclusive. Use only one of them.
    #   caCert: "-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----"
    #   # caCert is for providing an inline PEM encoded CA certificate
    #   # caFile: "/path/to/ca/cert.pem"
    #   # caFile is for providing a path to a file containing a PEM encoded CA certificate

    # Optional: Number of worker threads for sending data
    # workers: 4

    # Configure the queue for buffering events
    queue:
      # Memory queue configuration
      type: memory
      maxLimitInMB: 1024  # Maximum queue size in MB
      # Note: The queue size can be lowered if needed, but it should not be necessary to increase it.
      # Optional: Action to take when queue is full
      # fullAction: "pause"  # Options: "pause" or "deleteOldest"
      # Optional: Flush timeout in milliseconds
      # flushTimeOutInMillisecond: 1000

    # Alternate disk queue configuration (uncomment to use)
    # queue:
    #   type: disk
    #   maxLimitInMB: 4096  # Maximum queue size in MB
    #   # Note: Changing the disk queue size requires a rewrite of the queue storage file.
    #   # Optional: Action to take when queue is full
    #   # fullAction: "pause"  # Options: "pause" or "deleteOldest"
    #   # Optional: Storage directory
    #   # storageDir: "/path/to/queue/storage"
    #   # Default storage directory is under the dataDirectory of the program.
  my_examplesink1:
    type: logscale
    url: "https://cloud.humio.com/"  # Replace with your LogScale instance URL
    token: "${INGEST_TOKEN}"  # Use environment variable for the ingest token
# Define the sources for data collection
sources:
  # Add your sources here. Examples include:
  # - file
  # - syslog
  # - wineventlog
  # - journald
  # - cmd
  # - unifiedlog

The MySinksName is a top level element which contains each of your sink configurations.

Table: LogScale Sink

ParameterTypeRequiredDefault ValueDescription
compressionsinkhumiocompressionoptional[a] 'auto' Specify the type of data compression, possible values: auto, none, gzip, deflate.
   Values
   auto
   deflate
   gzip
   none
compressionLevelsinkhumiocompressionleveloptional[a] 0 Specify the level of compression where 1 is best speed and 9 is best compression, if set to undefined or 0 the default value for the compression algorithm, specified in compression, is applied.
maxBatchSizeintegeroptional[a]   Specifies the max size of batch (default 16MB) and works along with the maximum events per request. The limits are also propagated to all the sources that reference the sink.
maxEventSizeintegeroptional[a]   Max event size. Specify the number of bytes. When using Syslog set this to 8388608 where possible to avoid truncation issues.
proxyproxyoptional[a] 'auto' Set to none or you can specify, if required, an override proxy configuration for the sink, possible values: system, none or a URL such as: http://127.0.0.1:3129 for a http proxy. The default is system, which will try to determine the appropriate proxy or fallback to none.
queuequeueoptional[a] type: memory Can be set to memory or disk when disk is specified the data is written in the dataDirectory/queue/sinkName/ by default.
tlstlsoptional[a] {} This object contains details on the PEM certificates. this section allows you to override the defaults. Only one of the options should be specified.
tokenstringrequired   Specify the Ingest Tokens for your LogScale repository or an environment variable.
typelogscalerequired   The sources block configures the sources of data that the log collector will send to Falcon LogScale.
urlstringrequired   Specify the url of your LogScale account for example https://cloud.humio.com.
workersintegeroptional[a] 4 Specifies how many workers to use to send to LogScale, under normal circumstances leave this at the default setting, for more information see Sink Workers.

[a] Optional parameters use their default value unless explicitly set.