Minimal Configuration Example Collecting File Data Source

This configuration is the minimal configuration needed to collect events from local log files.

yaml
dataDirectory: data
sources:
  apache_logs:
    type: file
    include: /var/log/apache/*.log
    sink: my_humio_instance

sinks:
  my_humio_instance:
    type: humio
    token: <ingest-token>
    url: https://cloud.community.humio.com

The sources section describes the data that should be collected, and the sinks section describes where those events should be sent. The sinks can be reused and are referenced by name in the source. The following documentation describes this file in more detail.

dataDirectory
dataDirectory: data

Data directory is a top level element which is required to define where the LogScale Collector will create its "database", e.g. checkpoints.json file. Note this is not required for remote configurations.

sources
yaml
sources:
  apache_logs:
    type: file
    include: /var/log/apache/*.log
    sink: my_humio_instance

The sources section defines the details related to your data source/s, you can define multiple sources by creating name elements like "apache-logs " that contain the configuration for each source, I.E type, details and the sink to be used.

  • apache-logs

    Each source must have a user speicifed name as top element in this case we have used apache-logs.

  • type

    The type of source in this case file but could be any of the supported Data Sources types.

  • include

    The file path to include when collecting files, this only applies to the source type file.

  • sink

    The name of the sink (destination for the collected data) you specified in the sinks section.

sinks
sinks:
  my_humio_instance:
    type: humio
    token: <ingest-token>
    url: https://cloud.community.humio.com

  • sinks the top level element which contains all the details on the sinks (data destinations) for your data collection, this element must contain the following sub-elements, you can have one or more sinks each defined by a name, like "my-humio-instance".

  • my_humio_instance

    Each source must have a top level name for the configuration of a sink, you will then reference this name in the sources section.

  • type

    The type of sink being used this must be set to humio.

  • token

    The ingest token for the LogScale repository to which you want to send data, for information on ingest tokens see Ingest Tokens

  • url

    The URL of your Losgcale account.

Note

You must set the url and token values that correspond to your LogScale instance and repository.